techy stuff (19)


Dev tooling with docker

This is the first in what will become a series of articles on development processes and techniques in use at my current place of work, Public Desire.  To start off with I’m going to take a run through our docker based development environment, then in future articles I might dip into our deployment processes and testing.  In my usual fashion I’m going to stay away from delving too deeply into the concepts and technical gubbins underlaying Docker and concentrate on the pragmatic business of getting a dev environment up and running.  If you’ve wanted to give docker a try, but found it a little intimidating give this a shot….




Chuwi Hi10 Pro keyboard / trackpad fix

Hi there, been a long time… nearly 6 months since I last posted anything.  Having a kid will do that to ya!

Anyway, this week I picked myself up a nifty little 2 in 1 tablet / laptop thingy from a chinese manufacturer I’d never heard of: The Chuwi Hi10 pro. For a grand total of £190 (£20 of which was import duty) I got this little beauty (after waiting a couple of weeks for delivery)

chuwi




Bannr.io

For the last couple of months I’ve been working on a platform to solve one of the problems I’ve seen repeated across numerous businesses in the ‘fast fashion’ ecommerce space.  The problem is how to update your homepage and site with the numerous promotions you’re running in a timely fashion without impacting site performance.  For the majority of businesses in this space, Magento is the platform of choice for the web facing side of the business.  Unfortunately Magento’s tools for this particular problem are lacking.  Smaller businesses tend to use CMS blocks with javascript timers to make sure their promotions show at the right time.  Moving up the scale, we have plugins for magento that allow cms blocks to be shown on a schedule.  And at the top end of the cost spectrum we have ‘personalization’ platforms that, as well as offering variant testing and personalization also offer the ability to schedule blocks of content.




Seeking new opportunities….

So, the time has come once again when I’m looking for new job opportunities.  As of the 1st of June I’ll be available for any fantastically interesting permanent opportunities in and around Manchester (Uk) or remote working.

you can view my linkedin profile here for a quick history of my experience, and if you want any further details contact me at jobs@tall-paul.co.uk for a copy of my resume.




MVC Architecture

This will be a slight departure from my usual posts about random bits of code and have more a focus on architecture.

In the last few years the world of development has seen quite a bit of upheaval as it adapts to new ways of working, the devops movement has, when implemented well, proven incredibly effective in increasing ‘throughput’ of code as well as offering opportunities for team empowerment and increased stability of the final product.  Unfortunately, that ‘when implemented well’ is sometimes a sticking point.  The idea behind devops is too integrate development and operations, to make operations (ie: deployments, infrastructure) more code driven and automated.  What happens when it’s implemented badly though is that a separate ‘devops’ team is setup within an organisation, the majority of the members being sysadmins with maybe one guy who knows how to write bash scripts.  I’ll cover devops in more depth in another post, but what I want to focus on in this post is the move towards software defined architectures, and how that movement can be harnessed effectively.




Play spotify through a different sound device (windows)

Just a quick post today about how to solve a common (maybe) frustration…

My setup when I’m working is that I sit at my desk in the kitchen, with spotify open to have some nice background music going on. However, my nice amp with its decent bass levels and all the rest is sat in the living room. That’s not really a problem, since I have one of these which allows me to play music from my laptop via bluetooth to the amp. However, the standard setup in windows means that you have one ‘default’ sound device, and that’s the one that spotify uses. So if I want my system sounds to come out of my laptop speakers (rather than, for example, conducting skype conversations via my amp) it means I have to play music through the laptop speakers as well.

Until I found this:

http://www.equalify.me/

With the simple addition of this rather wonderful piece of (free update: equalify is no longer free according to comments from readers, it now costs $5) software I can happily play spotify through my amp via bluetooth while keeping everything else through my ‘default’ laptop speakers.

Here’s how it works. First, in windows select your default audio device. Remember, this should be the device you want everything except spotify to play through.

The easiest way to do this is to right click on the volume mixer icon in your taskbar and select ‘playback devices’. You should see a window like this:

Untitled

right click on the device you want to use as the default (if it isn’t already set as such) and select ‘set as default’. Again, remember this is the device you want to play everything except spotify through.

Now, download and install equalify from here: http://www.equalify.me/?page=download

Once it’s installed, fire up spotify and you should see a new icon next to your search bar, like this: 1

Click it, and the equalify window should open at the top left of your spotify window. Right click on the equalify window and wight click to activate the context menu. Select ‘options’ and then ‘change sound device’ like so:

Untitled

From the window that pops up, use the drop down to select the audio device you want to use and then click ‘Set New Output Device’:

Untitled

Now restart spotify.

And that’s pretty much it, spotify should now play through the device you selected.

As well as allowing playback through different audio devices, equalify also acts as an equaliser… if you like that sort of thing.




DIY security keypad

After my couple of hours of tinkering with python yesterday (see here) I started pondering how to expand on what I’d done.  Some sort of keypad would be handy, so I could just tap in a code and disable my cameras…. and then I’d need something to run the code on and connect the keypad to….Oh, and some way of indicating whether the cameras are currently enabled or disabled would be nice.

Luckily, I still had my trusty raspberry pi sat around doing nothing, so that solved the platform issue quite nicely.  The debian install for the pi already comes with python, so it was trivial to get my code up and running…. now I just needed a keypad.  I did look at something like this:

£10 from amazon might seem like a bargain, but I already had a spare usb keyboard sat around.  If only I could use that.  Well, that’s where some hacking came in handy:

2013-05-28 19.31.43

 

 

Somewhat astonishingly, the numpad bit of the keyboard still worked after my delicate surgery (I got lucky and happened to have a keyboard that responded well to being cut in half.  your mileage may vary).  So now we have a platform for the code to run on, and a number pad to enable or disable the cameras…it would be nice if we could determine whether the cameras are enabled or disabled though.  Handily, we can!  Pretty much every keyboard has a selection of LEDs on it to indicate the status of the ‘locking’ keys (caps lock, num lock and scroll lock) and my hastily hacked up masterpiece was no exception.  And it turns out it’s fairly straightforward to control these lights in python (although you do need to have root privileges to do so.  Why?  because linux that’s why) .

So, with a bit more googling and some slapping together of code here’s how it all works.

The pi is set to login automatically on boot (using the info here) and starts the python script (source below).  The script then queries motion for the current status, and sets the scroll lock LED accordingly, before waiting for the user to enter a passcode.  If the passcode is incorrect the lights on the keyboard flash on and off, before returning the user to the ‘enter passcode’ bit again.

Here’s a couple of images of the ‘mounted’ number pad (sellotape is brilliant!):

cameras off / cameras on.

 

2013-05-28 21.04.36 2013-05-28 21.04.43

As well as some tidying up, I want to do some more work on the python script (I’d quite like it to poll the cameras while waiting for input, and flash the capslock key as a ‘heartbeat’ indicator) but here’s what the code looks like so far:

 

#!/usr/bin/python
import getpass
import requests
import os
import fcntl
import time
import pwd
import grp

KDSETLED = 0x4B32
SCR_LED  = 0x01
NUM_LED  = 0x02
CAP_LED  = 0x04

def get_status():
        response = requests.get(url= host+"/1/detection/status")
        if response.status_code == 200:
                if "PAUSE" in response.text:
                        return 0
                else:
                        return 1
        else:
                return -1

def toggle_status(current_status):
        if current_status == -1:
                return
        if current_status == 0:
                path = "/detection/start"
                out = " started"
        else:
                path = "/detection/pause"
                out = " stopped"
        for x in range(1,camCount+1):
                response = requests.get(url= host+"/"+str(x)+path)
                if response.status_code == 200:
                        print "Camera "+str(x)+out
                else:
                        print "Error with camera "+str(x)

camCount = 2
password = "PASSWORD"
host = "http://HOST:PORT"
console_fd = os.open('/dev/console', os.O_NOCTTY)
enabled = SCR_LED | NUM_LED
disabled = NUM_LED
error = CAP_LED
all_off = 0
all_on = SCR_LED | NUM_LED | CAP_LED

while True:
        os.system('cls' if os.name=='nt' else 'clear')
        status = get_status()
        if status == 0:
                print "Motion detection disabled"
                fcntl.ioctl(console_fd, KDSETLED, disabled)
        else:
                if status == 1:
                        print "Motion detection enabled"
                        fcntl.ioctl(console_fd, KDSETLED, enabled)

                else:
                        print "Error determining status"
                        fcntl.ioctl(console_fd, KDSETLED, error)
        input = getpass.getpass("Enter the passcode: ")
        if input == password:
                toggle_status(status)
        else:
                fcntl.ioctl(console_fd, KDSETLED, all_on)
                time.sleep(2)
                fcntl.ioctl(console_fd, KDSETLED, all_off)
                time.sleep(2)



Spotify on Debian ‘Wheezy’ (testing)

Hello! Long time no post…. I’ve recently(ish) started working from home as head of IT for a small web design firm… which means I get to use a linux desktop again.  I’ve settled on Debian testing (which is currently ‘wheezy’) but have found a few niggles (always the way) so my next few posts will be discussing how to set up various things on Wheezy.  Just as an aside, I’m using xfce4 as my desktop environment..I like to keep things fairly clean and simple.

If you try and install spotify following the instructions given here:

http://www.spotify.com/uk/download/previews/

You’ll hit an error complaining about libssl… basically the spotify client is compiled with a hard link to libssl 0.98, which isn’t available from the debian testign repositories… since it’s been updated to 1.0.

To fix this, download the correct package for your architecture (so amd64, i386 etc) from here: http://packages.debian.org/squeeze/libssl0.9.8

Install the package using dpkg (double click on the downloaded file, or run dpkg -i /path/to/file.deb from the command line)

then follow the instructions from spotify:

# 1. Add this line to your list of repositories by
#    editing your /etc/apt/sources.list
deb http://repository.spotify.com stable non-free

# 2. If you want to verify the downloaded packages,
#    you will need to add our public key
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 94558F59

# 3. Run apt-get update
sudo apt-get update

# 4. Install spotify!
sudo apt-get install spotify-client
And that's pretty much it.  Enjoy!



Setup redmine / artifactory / hudson on fedora

Just a quick run through of what I’ve been doing for the last few days, namely setting up our build / development server.  It’s running Fedora, and the end goal was to have redmine, artifactory and hudson all on the same server, all running on port 80.  Once it’s installed and running we should be able to run automated builds (using hudson) when we commit to subversion and have those builds available from maven (using artifactory) for our Servicemix installation on another server.

It’s been a fun ride, redmine in particular was loads of fun to get going since I’ve never really setup a rails application before.  Anyway, on with the run through.  Please note that this is a very basic guide, and assumes you’re comfortable with the linux command line and so on, it’s not really for beginners.

Server

Install fedora, ensure apache / mysql are installed and working, also install the latest JDK (guide here)  and ensure your JAVA_HOME environment variable is set correctly.

Redmine

Download & install redmine following instructions here (for this example assume you installed redmine into /var/redmine).  You might have problems with bundler reporting some missing libraries, specifically ‘Rmagick’.  this should help with that.

Install passenger from here

make a symlink from a folder in your apache webroot to the ‘public’ folder of wherever you installed redmine:

ln -s /var/redmine/public /var/www/html/redmine

add an .htaccess file in the ‘public’ folder of redmine containing:

PassengerAppRoot /var/redmine #this should be the physical location of the redmine root directory on the filesystem
RailsBaseURI /redmine #this is the URI you want redmine to be accessible at, should match the symlink in /var/www/html/

Now in your web configuration folder (/etc/httpd/conf.d/) add a new ‘redmine.conf’ (or however you configure apache, it varies across versions of fedora I think) containing this:

Alias /redmine /var/www/html/redmine
<Directory /var/www/html/redmine>
 Options Indexes ExecCGI FollowSymLinks -MultiViews
 AllowOverride All
 Order Deny,Allow
 Deny from None
 Allow from All
</Directory>

Finally we need to set redmine to start in production mode.  Edit /var/redmine/config/environment.rb and uncomment or add the following line at the top:

ENV['RAILS_ENV'] ||= 'production'

And that should be it… restart apache and you should be able to access redmine at:

http://yourhost/redmine

Artifactory

Download and install the ‘rpm installer’ for Artifactory using the instructions here

before setting up apache we need to change the port the ‘ajp’ connector runs on, as our hudson installation will steal this later.

edit the file: /opt/artifactory/tomcat/conf/server.xml and find the section that looks like this:

<Connector port="8009" protocol="AJP/1.3"
 maxThreads="500" minSpareThreads="20" maxSpareThreads="150"
 enableLookups="false" disableUploadTimeout="true"
 backlog="100"/>

change the connector port to “8010” and save the file

In the apache configuration, setup reverse proxying as follows (/etc/httpd/conf.d/artifactory.conf)

ProxyPreserveHost on
<Proxy *>
 Order deny,allow
 Allow from all
</Proxy>
ProxyPass /artifactory ajp://localhost:8010/artifactory retry=0
ProxyPassReverse /artifactory/ http://YOURHOST/artifactory/
ProxyRequests Off

Start artifactory (/etc/init.d/artifactory start) and restart apache (/etc/init.d/httpd restart) and that’s it really.  Artifactory is now available at:

http://yourhost/artifactory

Hudson

Download and install Hudson using the method described here

Create a proxy config file for apache (/etc/httpd/conf.d/hudson.conf) containing the following:

ProxyPreserveHost on
<Proxy *>
 Order deny,allow
 Allow from all
</Proxy>
ProxyPass /hudson http://localhost:8080/hudson retry=0
ProxyPassReverse /hudson http://10.10.10.42/hudson
ProxyRequests Off

Finally, we need to tell hudson that it’s running on /hudson otherwise all the css / js links break.

edit: /etc/sysconfig/hudson

and edit / change the following line (it’s near the bottom of the file):

HUDSON_ARGS="--prefix=/hudson"

Now start hudson (/etc/init.d/hudson start) restart apache and we’re done.  Hudson is now available at http://yourhost/hudson

 

 

 

 

 

 

 

 

 




FUSE ESB Development: the build environment

The first thing we need to do when starting to work with fuse ESB is set up our build environment.  For this we’ll be using Eclipse, with the FUSE IDE plugin and maven. I’ll also cover downloading and installing Servicemix, and then generating our first FUSE project.

I’ll be using a windows box for the majority of this tutorial, but it should be fairly straightforward to get this working on your favoured flavour of *nix if you’re so inclined.

There’s a lot to get through here so lets get started.

Maven.

Download the latest version of maven from here: http://maven.apache.org/download.html

Extract the downloaded zip somewhere sensible (I use c:\program files\maven)

To make sure we can use maven from the command line we need to add it to our ‘Path’ environment variable.  On windows 7  you can do this by right clicking ‘Computer’, selecting ‘advanced system properties’ then click the ‘environment variables’ button.  Once the box opens scroll down until you find ‘Path’ under system variables, click edit and add ‘c:\program files\maven\bin’ to your string.

That’s maven setup, lets move onto something a little more challenging.

JDK installation

Before we can install our FUSE products we need to grab the Java Development Kit (JDK) and ensure that Eclipse and servicemix know where to find it by setting our ‘JAVA_HOME’ environment variable.

grab the JDK from here: http://www.oracle.com/technetwork/java/javase/downloads/jdk-6u25-download-346242.html and run the installer.

Once it’s installed you need to go back into the ‘Environment variables’ box we were in during the Maven installation and add a new variable called ‘JAVA_HOME’ pointing at the Directory your just installed the JDK into (by default it should be something like ‘C:\Program Files\Java\jdk1.6.0_25’).

Eclipse

Download Eclipse Helios from here: http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/heliossr2

Personally I’ve had no problems using the later ‘Indigo’ version of eclipse, but FUSE recommend Helios for working with their IDE so we’ll stick with that.  Run the installer as usual.  Once it’s installed we need to get the FUSE IDE plugin.

Go here: http://fusesource.com/products/fuse-ide/ to download it.  You need to register as part of the ‘Fuse community’, then add the update site listed on that page to your eclipse installation to add the IDE plugin.

Once that’s done the final part of the Eclipse installation is to make Eclipse use our previously installed maven repository rather than the one that comes bundled with Eclipse.  In the Eclipse main menu open ‘Window’->’Preferences’.  Then select ‘Maven’->’Installations’.  Click ‘Add’ and browse to your maven directory (in my case it’s ‘c:\program files\maven’).  Click OK and you’re done.

ServiceMix

For our ServiceMix installation I’m going to use the packages provided by FuseSource as ‘Fuse ESB#.  You can get the latest version from here: http://fusesource.com/products/enterprise-servicemix/.

Download the latest Zip file and extract it somewhere obvious (c:\servicemix for example).

And…that’s pretty much it, you should now be able to start Servicemix from the command line by changing to the servicemix directory and running the command ‘bin\servicemix.bat‘.  If everything goes well you should see something like this:

Generating our first project

Now everything’s setup lets try generating and deploying a simple test project.

To generate our project structure we can use the maven archetype generator as follows:

First create a new, empty directory where we’ll be working (I’m using c:\testproject)

Next run the command ‘mvn archetype:generate‘. this will prompt maven to download the latest list of archetypes and display them as a huge list.  Luckily we can filter this list if we know the name of the archetype we want. Type ‘karaf-blueprint-archetype‘ to return our filtered list, which should have 1 entry.  press 1 to select this entry and then from the list of versions that follows select the latest one (at time of writing this is 2.2.7).

You’ll now be prompted for some information about the project, use the following (or replace with your own values as required):

Once you’ve entered the details a simple ‘Y’ will confirm them and generate our project.

Import project into eclipse

The next stage is to import our project into Eclipse.  In the Eclipse main menu select ‘File’->’Import’, then from the import box scroll down and select ‘Maven’->’Existing Maven Projects’.

In the Import window browse to the location of your test project (notice maven has generated it in a subdirectory of our root directory) as shown:

Click ‘Finish’ to import the project.  You may get a ‘NullPointerException’ here for some reason, but the project should still import.

Configure a ‘Hello World’ Route

Now let’s make our project actually do something.

If you open up your project structure, you should see something like this:

There’s a few files and folders here to take note of, and we’ll go into these in more detail in further installments of this series, but for now let’s just jump in and start hacking around.

First of all we can remove the sample java beans that Maven gives us, so delete the two files under your java class path.  In my case this is ‘test-project\src\main\java\uk\co\tall-paul’.

We can also get rid of the sample camel context (in the file ‘my-service.xml’) and replace it with our own.  So remove that file and create a new one in the same place called ‘hello-world.xml’ containing the following:

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:camel="http://camel.apache.org/schema/blueprint"
    xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0"
    xmlns:cxf="http://camel.apache.org/schema/blueprint/cxf"
    xsi:schemaLocation="
       http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
       http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/spring/camel-blueprint.xsd">       
	
	<camelContext id="hello-world-context" xmlns="http://camel.apache.org/schema/blueprint">
	
	<route id="hello-world-route">
		<from uri="timer:test?period=5000"/>
		<log message="hello world!"/>
		<to uri="mock:result"/>
	</route>
		
	</camelContext>
</blueprint>

For more info on what that xml actually does have a look at the camel website, and specifically the information abou the various components which you can find here: http://camel.apache.org/components.html

This simple route basically uses a timer to fire off a message every 5 seconds, then logs ‘hello world’ to the console.

Build the Project

To build the project we can run maven from the command line or from within eclipse.

From the command line, navigate to the project folder (c:\testproject\test-project) and run the command ‘mvn clean install‘ (you can find more information about various maven commands here: http://maven.apache.org/maven-1.x/reference/command-line.html , but really ‘mvn clean install’ is all you’ll need for now).  If all goes well you should (eventually) see something like the following:

From Eclipse you can do pretty much the same thing by right clicking on your project and selecting ‘Run as’->’Maven install’ as shown here:

Our project has now been compiled and installed into our maven repository, which you can see if you browse to the location of the repository (by default this is at c:\users\YOURUSERNAME\.m2\repository):

Deploy the project into servicemix

Thanks to the integration of maven into servicemix, deploying our project is trivial using the ‘install’ command withing servicemix.

From your servicemix consolerun the following command:

install mvn:\uk.co.tall-paul\test-project\1.0-SNAPSHOT

Hopefully you should be given a bundle ID, something like ‘bundle ID: 200’.  Now type the following command (substituting the bundle ID you were given for the 200 shown here)

start 200

And that should be it.  As long as you didn’t see any error messages, your bundle should now be running.  To see the output from our route type the following commands (the first clears the log file for clarity, the second displays the current log)

log:clear
log:tail

after a few seconds you should see something like this:

You can see that every 5 seconds our timer is going off and logging our message to the console. You can also see here the name of the component the message originated from (remember we put ‘test’ as the first parameter for our timer component) and the route in which the message is running (here it’s ‘hello-world=route’), which indicates the importance of naming your routes and components sensibly.

Conclusion

Hopefully this brief tutorial has shown how easy it is to get up and running with Fuse ESB development.  Next time we’ll look at how to use the environment we’ve set up here to do some real integration work.