Carl's Blog

Motivation

My newly ordered Arduino Uno came in the post today so I got to start building with it!

An upcoming sensing deployment needs a way of sensing gas usage, so I’m building a basic sensor to measure it. Luckily, most gas meters have an LED pulse which goes off every so often and we can measure that. The one in my house informs me the light will pulse every 10 dm^3 of gas, so every time the second number from the right on the meter ticks the light will pulse. Failing this, the 0 on the right most dial is very reflective, and an LED can be used to reflect light from it.

As the extra bits I need to finish the sensor hasn’t come yet, namely the SD card shield and a battery pack, I can’t lock the sensor in my gas cupboard and give it a test, so for the time being I’m turning off my computer screens and lights, then flashing a red head torch at it to simulate a pulse.

Code

After a bit of internet trawling, I found an ace site laying down how to do the wiring for a digital light sensor (http://www.ladyada.net/learn/sensors/cds.html#bonus_reading_photocells_without_analog_pins). From that, I added in a method which took a rolling average of the last 5 measurements and outputted if the measured light was brighter than 110% of the average (not exactly statistical, but I intend for these to be locked in a dark box…).

Here’s what I have so far:

/* Based on: Photocell simple testing sketch. 
Connect one end of photocell to power, the other end to pin 2.
Then connect one end of a 0.1uF capacitor from pin 2 to ground 
For more information see www.ladyada.net/learn/sensors/cds.html */

// Number of measurements for averaging
const int AVERAGE_LENGTH = 5;

int photocellPin = 2;     // the LDR and cap are connected to pin2
int photocellReading;     // the digital reading
int ledPin = 13;    // you can just use the 'built in' LED

// for the circular buffer
int lastReadings[AVERAGE_LENGTH];
int average = 0;
int counter = 0;

void setup(void) {
  // We'll send debugging information via the Serial monitor
  Serial.begin(9600);

  // Initialise the array 
  for(int i=0;i<AVERAGE_LENGTH;i++) {
    lastReadings[i] = 0;
  }   
}

void loop(void) {
  // read the resistor using the RCtime technique
  photocellReading = RCtime(photocellPin);

  // Calculate rolling average
  ++counter %= AVERAGE_LENGTH;
  lastReadings[counter] = photocellReading;
  calcStats();
  int bound = average - (average/10);

  // Glorious debug
  Serial.print("Av = ");
  Serial.print(average);
  Serial.print(" = [");
  for(int i=0; i<AVERAGE_LENGTH;i++) {
    Serial.print(lastReadings[i]);
    Serial.print(",");
  }
  Serial.print("] ");
  if (photocellReading < bound) {
    Serial.print(" . RCtime reading = ");
    Serial.println(photocellReading);     // the raw analog reading
  } else {
    Serial.println();
  }

  delay(100);
}

// Calculates the new mean based on the last 20 measurements 
int calcStats() {

  // average
  average = 0;
  for(int i=0;i<AVERAGE_LENGTH;i++) {
    average += lastReadings[i];
  }
  average /= AVERAGE_LENGTH;
}

// Uses a digital pin to measure a resistor (like an FSR or photocell!)
// We do this by having the resistor feed current into a capacitor and
// counting how long it takes to get to Vcc/2 (for most arduinos, thats 2.5V)
int RCtime(int RCpin) {
  int reading = 0;  // start with 0

  // set the pin to an output and pull to LOW (ground)
  pinMode(RCpin, OUTPUT);
  digitalWrite(RCpin, LOW);

  // Now set the pin to an input and...
  pinMode(RCpin, INPUT);
  while (digitalRead(RCpin) == LOW) { // count how long it takes to rise up to HIGH
    reading++;      // increment to keep track of time 

    if (reading == 30000) {
      // if we got this far, the resistance is so high
      // its likely that nothing is connected! 
      break;           // leave the loop
    }
  }
  // OK either we maxed out at 30000 or hopefully got a reading, return the count

  return reading;
}

Next Steps

Well apart from the hardware I need, so issues need to be addressed which may or may not require extra hardware – as I’ve just thought of them.

• DateTime equivalent object for when I register a pulse
• Work out how long these will last on battery
• Can I set an interrupt to go off when a digital value hits a threshold? Or does this require analogue input? If I can it would massively save on battery as no polling! But, it may require fiddly per-house calibration, which the brute force method ignores
• Laser/3d Printed box and some form of mounting which will let me attach to anything. Probably going to be velcro.

Here’s a video of it working:

I found this cool graphing JS library which has a wordpress plugin! It’s called JSXGraph and is rather nifty!. Here is an example graphs showing Facebook users over time.

Here’s the code:

<jsxgraph width="600" height="200" box="box">
var brd = JXG.JSXGraph.initBoard('box',
            {boundingbox: [-50, 900, 3000, -150],
             keepaspectratio:true,
             axis:true,
             grid:0,
             showNavigation:false});

brd.suspendUpdate();

//Points for graph
var p = [];
  p[0] = brd.create('point', [0,0], {style:6,name:""});
  p[1] = brd.create('point', [1665,100], {style:6,name:"100"});
  p[2] = brd.create('point', [1890,200], {style:6,name:"200"});
  p[3] = brd.create('point', [2050,300], {style:6,name:"300"});
  p[4] = brd.create('point', [2193,400], {style:6,name:"400"});
  p[5] = brd.create('point', [2359,500], {style:6,name:"500"});
  p[6] = brd.create('point', [2527,600], {style:6,name:"600"});
  p[7] = brd.create('point', [2672,700], {style:6,name:"700"});
  p[8] = brd.create('point', [2787,800], {style:6,name:"800"});

//Line
var graph = brd.create('curve',
              brd.neville(p),
              {strokeColor:'red',
               strokeWidth:5,
               strokeOpacity:0.5});

//Labels
xtxt = brd.create('text',[1400,-110, 'Days Online'], {fontSize:12});
ytxt = brd.create('text',[10,850, 'Millions of users'], {fontSize:12});

brd.unsuspendUpdate();
</jsxgraph>

So I’ve migrated my blog from blogger to wordpress, with the advent of google+ this could have been a premature move, but wordpress is just *nicer*.

Some major points about this migration.

1. From Google’s servers to my own
2. Want to have support for multiple wordpresses
3. WordPress gets things via FTP (eurgh)

So, point 1 and 2.

I made a directory in /srv for the numerous wordpresses, and then created a mysql database ready for the blogs (WP lets you have multiple blogs on the same database, by having different prefixes). Due to wanting to have multiple users and having the FTP features, I decided that this prefix would define the internal blog name. So for example, lets make a blog with the prefix ex.

1. create the directory exwordpress
2. make sure the directory is owned by the http
3. set permissions to 775, via sudo chmod -R 775 .
4. grab the wordpress tarball and extract
5. configure any traffic for your blog domain to go to /srv/[wordpress directory]/exwordpress/

Load up your page, and configure the wordpress to point to your database, and voila, your basic wordpress set up is done.

Now, I wanted to import my blogger content, so on the dashboard, tools, import, blogger … ahh I need to install a plug-in. Oh, it needs ftp access to my server …

On to point 3

I used vsFTP, which required some fiddling with PAM. There is a sample config on the wiki page which works out of the box. If you want to test just ftp to your server using your virtual user credentials and try and create a temporary directory. If you can, job done.

So, I finally get the blogger content imported, which is fine, but for a few minor issues.

• Every title, and the content, is preceded by a single “>”
  • Hey, if it is open source, I’ll see if I can find a fix …
• tags are converted to categories
  • Which isn’t that much of an issue with the tag<->category converter

So, conversion done, just a pity that the only way to fix the conversion bug was to manually edit my posts.

This was performed on Archlinux with lighttpd 1.4.28 and rails 3.0.3

Prerequisites

Required packages:

• lighttpd,
• fcgi,
• ruby,
• and their dependencies…

Ruby Setup

Required gems:

• fcgi,
• bundler

(if you are behind a proxy, the magic gem command is :

# gem install GEM -r -p "http://[PROXY_URL]:[PROXY_PORT]"

)

Once you have that you need to create a “dispatch.fcgi” script to do all the rails magic. I found an example one at http://stackoverflow.com/questions/3296206/rails-3-and-fcgi .

#!/usr/bin/ruby

require 'rubygems'require 'fcgi'

require_relative '../config/environment'

class Rack::PathInfoRewriter  

  def initialize(app)
    @app = app
  end

  def call(env)
    env.delete('SCRIPT_NAME')
    parts = env['REQUEST_URI'].split('?')
    env['PATH_INFO'] = parts[0]
    env['QUERY_STRING'] = parts[1].to_s
    @app.call(env)
  end
end

Rack::Handler::FastCGI.run  Rack::PathInfoRewriter.new(YOUR_APP_NAME::Application)

Running a “bundle install” from your app root will make sure all the necessary gems are available for local use. Follow these instructions and run “ruby public/dispatch.fcgi”, if you get no errors, voila!

Lighttpd Setup

Now, to set up lighttpd you need to merge this with your config:

server.modules   += ( "mod_fastcgi", "mod_rewrite" )

$HTTP["host"] == "localhost" {        

  server.document-root       =   "/path/to/your/app/public/"

  server.dir-listing         =   "disable"
  server.error-handler-404   =   "/dispatch.fcgi"

  fastcgi.server             =   (
                                   ".fcgi" => (
                                     "localhost" => (
                                       "min-procs" => 1,
                                       "max-procs" => 1,
                                       "socket" => "/tmp/ruby-beholder.socket",
                                       "bin-path" => "/path/to/your/app/public/dispatch.fcgi",
                                       "bin-environment" => ( "RAILS_ENV" => "development" )
                                       )
                                     )
                                   )
}

A quick “sudo /etc/rc.d/lighttpd restart” and a check of the error logs will tell you if it has worked

To all those interested, here is a copy of my email with my solution.

To whom it may concern,

Decoded this from the last challenge!

Congratulations  youve found and completed the REAL challenge. Your win
code is  cyb3r=s3cur1ty*ch@ll3nge+26-07-2010.

Please email this code to our team at
media@cybersecuritychallenge.org.uk. If youre the first person to do so,
and can prove you meet the eligibility criteria (British citizen
currently resident in the UK) we will be in touch to advise how to claim
your prize. Well done and good luck in the Cyber Security Challenge
competitions taking place throughout the rest of the year

Figured I would go through the process:

Task one

Copied the data into a file.

Created the following perl script:

#!/usr/bin/perluse MIME::Base64;

while(<STDIN>){    print decode_base64($_);}

I then piped the data through the script into another file

$ cat data | ./base64.pl > output

At a hunch, checked if it was an image.

$ evince output

Voila!

Task two

Noticed the squiggles, zoomed in a lot and noticed it covered 2 pixel strips, so converted the image to a bitmap, then created a very hacky C file to grab the data (after checking the wiki page for the  file format). It is quite hacky, but combined with an online binary to ascii converter, worked well. The C file was as follows:

#include <stdlib.h>
#include <stdio.h>

void * imgdata;int datao(int x, int y);

int main(int argc, char *argv[]){

  FILE * fp;
  int w, h, o, off, i;
  int r, g, b, re;

  //Read binary data
  fp = fopen("output.bmp", "rb");

  fseek(fp, 0x0a, SEEK_SET);
  o = getc(fp);
  printf("1\n", o);

  //First
  off = o + 1052 * 173;
  fseek(fp, off, SEEK_SET);
  for(i = 0; i < 350; i++){
    r = getc(fp);
    g = getc(fp);
    b = getc(fp);    

    if (r+g+b > 600)
      re = 1;
    else
      re = 0;

    if((i+1)%8)
      printf("%d", re);
    else
      printf("%d", re);
  }  

  printf("\n2\n", o);

  //section section
  for(i = 3; i <= 172; i++){
    off = o + (1052 * i) + 1047;
    fseek(fp, off, SEEK_SET);    

    r = getc(fp);
    g = getc(fp);
    b = getc(fp);    

    if (r+g+b > 600)
      re = 0;
    else
      re = 1;

    if((i+1)%8)
      printf("%d", re);
    else
      printf("%d", re);
  }

  printf("\n3\n", o);

  //third section  

  off = o;
  for(i = 349; i >= 0 ; i--){
    fseek(fp, off + (i*3), SEEK_SET);
    r = getc(fp);
    g = getc(fp);
    b = getc(fp);    

    if (r+g+b > 600)
      re = 0;
    else
      re = 1;

    if((i+1)%8)
      printf("%d", re);
    else
      printf("%d", re);  

  }
  printf("\n4\n", o);

  //forth section  

  for(i = 172; i >= 3; i--){
    off = o + (1052 * i) + 3 ;
    fseek(fp, off, SEEK_SET);    

    r = getc(fp);
    g = getc(fp);
    b = getc(fp);    

    if (r+g+b > 600)
      re = 0;
    else
      re = 1;

    if((i+1)%8)
      printf("%d", re);
    else
      printf("%d", re);
  }

  return 0;
}

//offst from data start

int datao(int x, int y){

  int width = 350;
  int height = 175;

  int offset = (y * width * 3) + x*3;
  return offset;
}

From this I got the following text:

Cyrnfr sbyybj guvf yvax:
uggcf://plorefrphevglpunyyratr.bet.hx/834wgc.ugzy
uggcf://plorefrphevglpunyyratr.bet.hx/834wgc.ugzy

Which after running through the following perl script:

#!/usr/bin/perl

use MIME::Base64;

while(<stdin>){  $_ =~ tr/a-zA-Z/n-za-mN-ZA-M/;  print $_;}

Which led me to …

Task three

This one was tricky. I looked for ascii values on the data, reversed, big endian, little endian, looked for 7bit data compacted into 8, 7bit and parity (odd/even) and then only looking at bits which  passed the test.

Early on I noticed there were no ’3′s in the data, but didn’t see the shocking clue.

Then I tried some basic frequency analysis, but as there were 56 unique entries it was probably a polyalphabetic cypher if it was. Tried the kasiski technique, ended up with random numbers, but tried none the less. I was convinced that the page name was a key, or something simple, like “cypher”. Alas, it was not true. Whilst looking at the binary for the 500th time looking for patterns of shifts which you could be employing, but finally I saw the solution. Turns out I was doing the rookie mistake of starting out complex.

Following is the ruby file used to decode the data:

class Cypherc

    attr_accessor :bytearray, :rawdata

    def initialize

        rawdata =  "68edcdec4e2c8eae8d2c8e2dedcd6e04d2042fedae52ceac04..."

        @rawdata = rawdata

        bytearray = []

        (0 ... rawdata.size/2).each do |i|

            bytearray << "0x#{rawdata.slice!(0,2)}"

        end

        @bytearray = bytearray

    end

    def differencekeysize(length)

        diff = []

        initial = @bytearray[0]

        (1...@bytearray.size).each do |i|

            if (i%length < 1)

                d = Integer(@bytearray[i]) - Integer(initial)

                initial = @bytearray[i]

                diff << d

            end

        end

        return diff

    end

    def asciicharacters

        ac = {}

        (0...@bytearray.size).each do |i|

            if(Integer(bytearray[i]) < 128)

                puts "#{i} : #{@bytearray[i]}"

                ac[i] = @bytearray[i]

            end

        end

        return ac

    end

    def paritycheck(byte, type = 1)

        #number of 1s

        binary = Integer(byte)

        tocheck = binary & 0x7f

        par = binary >> 7

        ones = 0

        (0..7).each { |i|

            ones += tocheck & 0x01

            tocheck = tocheck >> 1

        }

        ok = false

        if(type == 1)

            if par == 1 and ones.odd?

                ok = true

            elsif par == 0 and ones.even?

                ok = true

            end

        elsif(type == 2)

            if par == 1 and ones.even?

                ok = true

            elsif par == 0 and ones.odd?

                ok = true

            end

        end

        return ok

    end

    def bitvalue(byte)

        return byte & 0x7f

    end

    def diffscan(index)

        diffs = []

        tocheck = @bytearray[index]

        (index+1 ... @bytearray.size).each{|b|

            diffs << Integer(@bytearray[b]) - Integer(tocheck)

        }

        return diffs

    end

    def ngrams(n)

        bins = {}

        (0...@bytearray.size - n).each { |i|

            key = @bytearray.slice(i, n).to_s

            if bins[key].nil?

                bins[key] = 1

            else

                bins[key] += 1

            end

        }

        return bins

    end

    def bincheck

        bins = {}

        ks = 10

        (0 ... rawdata.size/ks).each {|i|

            bins[i] = rawdata.slice(ks*i, ks)

            t = []

            (0 ... bins[i].size/2).each do |j|

                t << "0x#{bins[i].slice(j*2,2)}"

            end

            bins[i] = t

        }

        comp = []

        bins.each{ |k,v| comp << bins[k][0] }

    end

    def shiftbits(byte)

        binary = Integer(byte)

        bottom = binary >> 5

        top = (binary & 0x1f) << 3

        return bottom ^ top

    end

    def decode

        @bytearray.each { |b|

            print "#{shiftbits(b).chr}"

        }

    end

end

c = Cypherc.new

c.decode

Those are my workings, 3 hours on the steg stuff probably, and much too long on the last challenge.

Looking forward to more challenges

Carl Ellis

http://www.jimhi.com

So I wanted to make a basic makefile for my tex documents, so I took the Makefile from http://www.acoustics.hut.fi/u/mairas/UltimateLatexMakefile/ and modified it to be amazingly simple.

Heres the Make file:

CC=
FLAGS=

LATEX= latex
BIBTEX= bibtex
DVIPS= dvips
PS2PDF= ps2pdf

SRC := $(shell egrep -l '^[^%]*\\begin\{document\}' *.tex | sed -e 's/.tex//')TRG = $(SRC).dviPSF = $(SRC).ps

all : pdf

dvi :  $(LATEX) $(SRC) && $(BIBTEX) $(SRC) && $(LATEX) $(SRC) && $(LATEX) $(SRC)

ps : dvi  $(DVIPS) $(TRG)

pdf : ps  $(PS2PDF) $(PSF)

clean :  rm *.bbl *.aux *.blg *.dvi *.log *pdf *.ps

Fairly simple and lets you choose your toolchain.

C

Ok, so you need to send certain keyboard events to an element to properly test a webpage, but stuck how to do it?

# First off create your Watir::IE object
ie = Watir::IE.new

# Then navigate to the page you are testing
ie.goto("mypage.myserver.com")

# Grab the descriptor of the element you wish to send a keyboard command to
# (for example, a text field)
element = ie.textField(:id, "myText")

# Now, I found that keyboard events only work if the Watir::IE window is on top and has
# focus, so for example if we wish to send a "Pageup" keyboard command to the element
# "myText" we would do so by
ie.bring_to_front
element.focus
element.container.focus
element.container.send_keys("{PGUP"})