Saturday 4 October 2014

Experiments with Pexpect

The Pexpect python module is used to start up and control new processes from python code.  For the Manchester CoderDojo I needed to incrementally get output from a spawned process over the course of a long run.  To test this I created a simple module that counted to 20 slowly:


"""
Count to 20 in one second steps
"""
from time import sleep

for i in range(1,21):
    print(str(i))
    sleep(1)

Next I created a python module to spawn off a process to call the counter and print the output.

"""
Call the slow-running counter and print the results.
"""

import pexpect

output = pexpect.run('python count-slow.py')
print(output)

The problem of course is that the run command blocks until all the output is produced.  The solution allowing you to  see the output as it appears is to use the pexpect.spawn() function.  This function to creates a sub-process object from which you can read one line at a time.  Pyexpect.readline() will block until it receives a line.  An empty string indicates the process has completed.

"""
Call the slow-running counter and print the results.
"""

import pexpect

child = pexpect.spawn('python count-slow.py')
next_line = child.readline()
while next_line != '':
    print(next_line.strip())
    next_line = child.readline()

Formatting Python Code for the Web with Pygments

I've been unsatisfied with Blogger's lack of easy handling for code formatting.  To fix this I installed Pygments.  I can now type,

pygmentize -f html -O noclasses count-slow.py

...and end up with output that I can paste into the Blogger HTML editor window.

The result is something like this:

"""
Count to 20 in one second steps
"""
from time import sleep

for i in range(1,20):
    print(str(i))
    sleep(1)




Friday 3 October 2014

Vagrant Working for the Manchester CoderDojo Minecraft Server Build

I've got a basic vagrant setup working for the Manchester CoderDojo Minecraft server build.  Instructions on the GitHub page.

Next task is to get the chunked responses working so that you can see the python output appear in the browser as the code is running.

Tip: Copying Large Files from Remote Desktop Sessions

I just had the problem of having to grab a large file from a Windows remote desktop session.  The usual cut-remote-paste-local magic wasn't going to manage the file size.  It turns out that RDP maps "\\TSCLIENT\" on the remote server to your local PC.  You can therefore use robocopy to bring the large file over locally using the restart mode in case of problems. 

For example to grab "big file.zip" from the current folder on the remote box to the c:\tmp folder locally run this:

robocopy /z /eta .  \\TSCLIENT\C\tmp "big file.zip"

Monday 29 September 2014

Vagrant and Docker Workflow

I've recently been looking into using both Vagrant and Docker to develop my projects.  This is the work flow that I've settled on...

  • I'll use Vagrant to support my development work
  • I'll use Docker as the mechanism for getting my projects hosted
  • Start a project with a Vagrant set up file that will launch a Debian vm and apt-get Docker installed on it
  • The Vagrant VM is used to build the code
  • The result of of the development code is to build a Docker image of the project
  • The Docker image is what is then used to deliver the project's services

As an example I created vagrant-docker-flask on GitHub to show how this works.

  1. Clone the git repo: git clone https://github.com/davegoopot/vagrant-docker-flask.git
  2. In the cloned directory run:  vagrant up
  3. You now have the dev machine image installed that you can connect to using: vagrant ssh
  4. Inside vagrant run: cd /vagrant/flaskserver/
  5. Then build the flask server image with: sudo docker build -t="local:flask" .
  6. Start up a container from the image with:  sudo docker run -d -p 5000:5000 local:flask python demo.py
  7. The flask server will run and be accessible from the original host machine at http://localhost:5000/
  8. To work on the code, you can now edit flask/demo.py, run the docker build and the docker run command again and see your changes

Monday 5 May 2014

Making the Raspberry Pi Email IP on Boot

I needed my Raspberry Pi to email its IP address to me when it booted in order to ease finding the IP address when running headless.  Here's how I did it:

  1. There are some really good instructions here: http://elinux.org/RPi_Email_IP_On_Boot_Debian
  2. In order for it to work I had to set up an application password for gmail since I'm using two-factor authentication to login
  3. Reboot and it sent me an email.  That was painless!

Twilio Interface for the Rube Goldberg Machine

Here's how I built the Twilio interface to the Rube Goldberg machine...

  1. Install the Twilio python libraries: sudo pip install twilio
  2. Set up a feature branch for Twilio: git checkout -b twilio-plugin
  3. Set up tests and code to load credentials from twilio.secret file
  4. From the secret file load the account_sid and auth_token
  5. Write the update_state() method to send an SMS with Twilio
  6. Decided not to unittest the SMS sending as this is just simple interface code with no logic
  7. Used the python code documented at:  https://www.twilio.com/user/account/developer-tools/api-explorer/message-create
  8. With a bit of Blu Tack, and some dominoes the system worked: http://youtu.be/Vj7IvwIN0r4