Let's get started with creating a Python Project Scaffold. What's important about this setup is that it's a repeatable process that you can do for every new project that you create. Let's suppose there's a GitHub repo and you check it out. In this case, it'll be in the Cloud9 environment, but to any environment, what are the things you need to do to be successful? The first thing is, I would recommend creating a Makefile. This Makefile will hold a series of recipes. Next up, we should create a requirements file. That'll be the next step here. It'll be called requirements.txt. After that, we can create some kind of a script. I'll start with hello.py file. Also, we can create a test file, so we will test our code. Then the final bit here would be that inside of this environment here, we could create a Python Virtual Environment. Really, these are the key components in order to test your code, get it set up so that later we can do continuous integration and we can do continuous delivery. Let's go ahead and do this in the AWS Cloud9 environment. Let's get started with creating this basic scaffolding for our project. The first thing that we'll need to do is create a new git repo. I'm gonna go ahead and go to GitHub. I'm going to create a new repo called scaffold, and then I'll put a description: This is a project scaffold for Python. Then a few things to point out here would be that it's always a good idea to add a readme file and also to add a gitignore file. I'm going to go ahead and select this and select the default Python template. What this does is make sure that you're attaining files like.pyc files or other files like eggs, don't show up inside of your Git repo, they'll automatically basically be ignored. Next, we'll go ahead and say Create repo. Now that I've got that set up, I can go back to my Cloud9 environment. I'm going to go ahead and select this icon code and I want to select SSH. This will allow me to do encrypted bidirectional communication and I don't have to have a password if I set up SSH keys. First let me just show you how this would work before I set up the SSH keys. I'll go ahead and copy this, go back to Cloud9 and I'll type in git clone. Here we go. Notice after it tries to connect, it says I can't do it. I have no way of connecting to this repo because I never told this environment how to communicate. What I'll need to do then is set up SSH keys. I'm going to go ahead and clear the screen and set up SSH keys and give those to my GitHub account. Step one is, let's type in ssh-keygen - t rsa. Notice that it'll put a public and a private key pair in this directory which is my home directory. Now what I can do if I scroll up a little bit, is you'll see this public key, that's the key that I want to print out into foreground and then upload to GitHub. Go ahead and run the word cat, which is basically printed out and print out that key. From here, I'll copy this string. This is not secret, I can actually put it on a website somewhere if I wanted to but I want to put this into GitHub. I'll go back to GitHub here and I will look at my profile and I'll go to settings. From here, I'll go to the section called SSH and GPG keys. Then I'll go ahead and create a new SSH key. I'll go ahead and paste it in here and then call this scaffold. It'll ask me for my password. Great. I'll go ahead and log this in and now if I go back to Cloud9, I should be able to check this out. Notice how if you use the up arrow or down arrow, you don't have to type in a command again. This is a great idea in general. Perfect. We've got this checked out. We're ready to build out this scaffold. The cd command is change directory into scaffold, and then I'll run an LS. You notice there's nothing inside of here. The next thing that I'll do is I'll use this touch command which will create empty file. What are the files I'll need initially? Well, I want to make Makefile, so I'll say touch Makefile. I'll also want a script file, so we'll make one called hello. I'll also want a test file, so I'll say test_hello.py. I'll also want requirements file, requirements.txt. This is pretty much all I need to get started. Then from here, what I can do as well is create a Python virtual environment. Again remember that the Python virtual environment isolates my code to a specific directory. I always like to name whatever the project is that I've created in GitHub the same name as a virtual environment. That way it's easier to remember. For example, if I type in git remote -v, you can see that the name of my project is called scaffold. Let's go ahead and create a virtual environment that's in my home directory that's invisible, and that way I don't have to accidentally check it in as well. If we go through here and we type in python3 -m, this stands for module. Create a virtual environment or use the virtual environment module, and put this in tilde, which is my home directory /.scaffold. Here we go. This is a good convention to use so that you never have to remember what it is that you called your virtual environment. Just call it the same thing as your git repo. Now that we've created that, I can source it. I'll go ahead and do that next. In the source that I would type in source tilde/.scaffold/bin/ activate. An activate is the shell script that activates the environment. Once I do this, if I type in which python, you can see that it'll be living now in this directory. It'll be living inside of tilde/.scaffold/bin/python3. Again, this is a really good best practice so that you don't have all these problems of there's packages that are conflicting with each other or the system python has problem. In general, I'll always recommend you do Python virtual environment. Now that we've got that set up, let's go to step 2, which is, let's look at that Makefile. If we look at this Makefile, we can do a couple things here. First, always make your Makefile use tabs, because Makefiles like tabs. Then what I'll do here is I'll just cut and paste, one that I've created earlier that's boilerplate. What you'll probably find is that you can use the same kind of Makefiles over and over again in your project. A lot of times it's good to just copy it and change it a little bit from a previous project. Let me walk through what all of these steps are. The first step is that I would do a pip install --upgrade pip. Pip is the python packaging tool that's actually included in a virtual environment. Notice if I go here and I say which pip, it'll show that it's actually using the python packaging tool in the virtual environment. That's yet another reason to use the virtual environment, is that you get the greatest version of pip's or the newest version that's installed with that version of python. I also like to install it though with an upgrade so that I always get the latest version. Then I use this requirements file to grab whatever packages my project will need. Next, there's a format section here that allows me to format my code using a formatting tool called Python black. This is optional, but I like to do it just so that my code is up to standard. Next, there's a lint section and I use a tool called pylint, and I tell it to disable two of the warnings that are extra verbose. These are recommended in configuration. What it'll do is it'll only give me then warning and error messages, which is what I really care about. Then I'll tell it to run the lint on this hello.py file. The fourth step here is testing. What I'll do is I'll use the pytest through a party library and I'll run python-m pytest. Give me some verbose output, give me a little bit of coverage here, and then run this test for this test code. That's really the basic structure and I would recommend this structure in every single project that you do. Now that we've got that set up, the next thing we can do is move on to the requirements, which will be the packages. I'm going to double-click this, and then now I can go back and decide what packages I want in my project. I would recommend probably just a few commands here that I typically find or libraries that I find in most projects. Pylint, as we discussed, is a popular linting tool, an alternative would be pyflake or flake8 There's also pytest, which is the testing library. There's click, which is accumalative library. There's black, which is the formatting library, and pytest-cov, which is a coverage tool that shows me how much of my code is got covered. Really, these five packages, I would recommend probably in all projects are generally going to be useful. I don't pin them and give them a specific number, like a particular version at first, but later, that could be a good idea, if I wanted to specifically call out a very specific version of the library. You could do that right here. But let's go ahead and save this. Now, we're ready to go. We can do an installation. This would be step 1. Let's go ahead and say make install. There we go. Then this will go through and look at the requirements file and install those packages as well as upgrading the version of pip, if there's an upgrade to be add. This typically will take, let's say, 30 seconds or so, depending on what you've got in your requirements file. Perfect. That was pretty quick here. The next step is, let's move on to creating some code. I'm going to go to a hello file. From here, a good one to start out with and it's probably not a bad idea to build the scaffolding out, for every project, is just something that gets things going. I'm going to build a add function, that accepts an x and a y and what it'll do is, it will return an x and a y. Then I can basically print this out. I can say print and I can say, this is the sum. We'll put in the x and then we'll put in the y, then we'll put in the final result, which will be the function that's returned with x and y. There we go. Perfect. That looks like we'll be able to print this out here. Notice that maybe by default here, we'll need to be told what the x and the y are? What I can do is put in a x is equal to one and y is equal to two. There we go. This should get things going here. Let's go ahead and say python, hello.py. There we go. This is the sum one and two. Notice that, in this particular example, I'm going to need to maybe get the result first. Let's go ahead and say results is equal to add and then I also need to change this to plus, and so then from here we can see that x and y will give us the result. To make it a little bit easier, I could actually say result. There we go. Perfect. If I go ahead and I run this, it should go 1, 2, 3, perfect. We've got something going here. I have a basic structure for my code. Now, that I've been able to demonstrate that, that works, the next thing that I could do is actually get lint working. How would I do this? Well, it's already built into the Makefile. It says lint, pylint. I can just say make lint. If things are successful, it says redefining x from outer scope, redefining y from outer scopes. I was a little bit naughty here, and I have some messy code. How could we actually fix this, if we wanted to make things a little bit easier? Well, what I could do is that, I could actually change around my code, so that lint wouldn't have a problem with it. What are the ways that I could do this? Well, one of the ways that we could do this is that, I could actually say in this particular scenario here, instead of defining x and y, I could actually do this. I could say this is the sum and I could do this, I can say one and two. Then this would clean it up because I wouldn't specifically put that in there. I would actually put the numbers inside, and now this would be acceptable because I'm not redefining those variables. Let's go ahead and run make lint here, and notice now I cleaned it up. So this is just generally a good way to make sure that you're not making silly mistakes in your code, and it checks the syntax without me even needing to write tests. So now that we've got that step up, right? So we've got step one and we've got step two here. Let's actually run this format tool. So I'm going to say make format, and you'll notice what will happen is it'll clean this up. So if I say make format, there we go, it said "one file formatted", and notice that it made some extra spaces here and it basically just made it look a little bit more pleasant to the eye. So it's an optional step, but I find it to be pretty useful. Now let's look at the last step here which would be to test our code. So in order to do that, I'm going to need to create a test file. Go ahead and look at this test file here, and say, "from hello, so I'll need to import my other code, import add." Next, I can make a test statement, so I'll say, "test_add ", and then it's actually pretty straightforward to test when you're using this PI test tool. I can assert that two values are what I expect. I can say for example, "add", which we know will add two numbers together. If I add one and two, that should equal three, right? I'll need to say "assert". So basically make sure that when I put in one and two in here, that it'll come back with three. Let's go ahead and do this, we'll say make test, and there we go. I've got a 100% test coverage. Now notice if I basically made an incorrect test, that it'll go in fail. There we go. We can say that, uh-oh, looks like there's a failure here, you know, four doesn't equal three. So I now have basically a complete setup where I can actually test everything at once. One other thing you can do with a makefile that's kind of neat is if you wanted to, you could actually run an all statement and run several of these commands all at once to really simplify how you would set up a project. So if I said all I could say, maybe I want to install this code, and then I want to lint it, and then I want to test it. Right? So if we go ahead and do that, if I say make all, it should run all those at the same time. Perfect. In a nutshell, this is a basic setup for a Python scaffolding. We've got everything ready to go locally, and we're ready to later setup continuous integration. The final thing to do, would be to push my code to GitHub. Let's go ahead and do that. I'll type in "git status", and you'll notice that there's four files here that need to be added. So because I have included that git ignore file, I can actually do this. I can say, "git add *", and it'll add all four at the same time. Then if I type in, "git status", you'll see they're now green. So they're ready to be pushed. Now I'll go ahead and I'll commit this. I'll type in git commit, which commits it locally. So adding initial structure, and then a notice that it's asking me to do some setup. The first time you set up your project, you'll need to do this configuration either in a file or manually, like I'm going to do it here. So I'll go ahead and put my name, and then I'll put in my email address here next. Let's go ahead and do that. Here we go, and the main reason for this is so that you get good tracking and you know what's happening in terms of metrics in your project, which is a good idea. So we'll go through here and we'll run this. Perfect. It opens up to the default editor, which in this case is NANO, and I'll type in a "Cntrl" key and then an O to write it out, and then I'll do a "Cntrl" X to exit. Then from here I can push it, and this is been committed locally. But when I push it pushes all of these files to GitHub. So let's go ahead and say git push, and here we go. How do we test this out? Well, I can go back to GitHub, and if I do a refresh, you'll see that all those files are located here. So It's all set up for me to do a Cloud-based continuous integration.
