That Coder Guy

Web, Mobile, Database and all things fun 
Written By: Davie Elliott

Sharing thoughts, insights, and knowledge...

By

Hiatus, Direction and Updates

hiatus2copy

Due to accepting a new job, I don’t think I will be able to write regular updates for this blog. However, I am currently building a workshop to build more tangible projects, which I shall fully document – and possibly even create a YouTube channel.

Some of the up-and-coming projects after completing the workshop are:

  • A smelting kiln (to fabricate my own parts)
  • A replica Enigma Machine
  • A mini-super computer
  • An SMS burglar alarm
  • And so many more…

Due to the length of these projects, updates to this blog will be few and far between.

By

A different direction…

The next instalment of “Diary of a web contractor” will now be postponed, due to unforeseen circumstances.

Instead I shall be setting my workshop up over the next month, and then this blog will chronicle some of the personal projects I’ll be doing over the coming year.

By

Diary of a web development contractor: Part 3

executive diary, a sleek cordless phone, and a cd-

Part 3: The legal bits & Re-establishing contact

In my last article, how to write your CV, and how to get noticed as a professional. In this part, I shall cover the legal stuff you need to look into.
 

The legal bits

 
As a contractor, you will no longer be a formal employee for a company, and any company you work for will not pay your PAYE taxes, your NI, or anything else other than your daily rate. This means you need to pay your own taxes, depending on the type of companies and how you want to do your taxes, you will have several options to consider.
 
First of all, you will need to decide how you will legally want to trade, generally there are three options; via an umbrella company, as a sole trader and as a limited company.
 

Umbrella company

 
An umbrella company will collect your money, and pay your taxes for you, all you do is pay them a monthly fee and then you can trade via the company. This seems like an attractive option, but there are a couple of caveats; firstly you cannot claim things back such as VAT, and some companies will only hire you if you trade as a limited company.
 

Sole Trader & Limited Company

 
These two options are very similar; you will need to pay your own taxes, and thus you will need an accountant, you can also claim your VAT back. However, again some companies will only hire you if you trade as a limited company.
 
Those are the trading options you will need to consider before getting your first contract, ideally you will need your company/umbrella setup as the hiring company will require this information when taking you on. You will also need to purchase indemnity insurance.
 

Indemnity Insurance

 
This is basically professional insurance, if you mess up and the company decides to sue you, then this insurance will cover you for the amount the company can sue you for. This is definitely something you want to purchase before getting your first contract, the monthly amount varies and a is usually based on your estimated yearly turn over.
 

Re-establishing contact

 
At 2-3 weeks before leaving you will want to contact all of the recruitment consultants whom you spoke to, so it’s a good idea to keep a copy of all the phone numbers and names of people who you have spoken to.
 
By the time I publish my next article I shall be at this stage, so the next few articles will be almost real-time.

By

Diary of a web development contractor: Part 2

executive diary, a sleek cordless phone, and a cd-

Part 2: The CV, Getting Noticed & Timing

In my last article, I covered the basics of what to think about when moving from a permanent job into contracting. The next two articles will cover each of the steps to take in more detail – again, I am only writing about what I have personally done, so don’t take these articles as gospel, just as advice from someone who has been there.
 

Your CV

 
When I looked for a new job, about 2 or 3 years ago, my CV only seemed to make agencies talk to me about mid-level development roles… now I seem to be getting calls about senior-level positions and more importantly, contractual roles – this is something that agencies have not called me about before, so I assume I must have written my CV to be more “contract-y”.

The first page of you CV, as pretty much anyone will tell you needs to be eye-catching, brief, and make the hiring staff want to talk to you after spending a few seconds looking at it. As a developer, I don’t really have a designer brain, so my CV doesn’t look pretty, but it does cover everything the hiring person would need to know about me, my skills and my experience.

Most contractors include a picture of themselves, I have no idea why, but I had to do this, since there is another person out there with the same name as me and also works (kind of) in the same industry as me. So it’s a good idea for hiring people to know what I look like, since nowadays, hiring managers tend to look people up on Facebook, Linked In etc…

Also, I think it’s a great idea to have a strapline for yourself, something which says what you specialise in, for myself I used “Systems Integrations Expert & Backend (middle tier) Developer”, which is exactly what I do. A personal profile also wouldn’t go amiss, something short and to the point, that describes you as a person and as a professional – but don’t take up too much valuable real estate with this.

One of the most important bits (particularly for someone in IT), is the skills section. This is the bit where you list all of your skills; what languages you know, what packages and frameworks you know, and also what methodologies you know. Not everything requires a version to be listed, I mean, who knows what version of C# or VB.Net were on; but a hiring manager would like to know that you’re using Visual Studio 2012, since that’s what they’re using in the company.

The last part to list is just a short list of companies and positions you have held in your career, this will give the hiring manager a brief understanding of if you will be a good match for the role. For example, having “Company XXX – Junior Developer – 2005 to 2014” on your CV when you’re applying for a Senior Software Architect isn’t going to work, neither is “Company YYY – Senior Software Developer – 2013 to 2014” when you’re applying for another senior developer role. Eventually this section will slowly be replaced with contractor roles you have held.

The subsequent pages of your CV need to go into more detail about the roles you held in each of the jobs you have listed on the first page. Ideally you want to write what you did in a project centric fashion, so instead of writing “Designed and Developed a system that does X, Y, and Z” you would write, “Lead Developer – Project X – This project allows users to do X and Y (Technologies used: C#, MVC, MEF. Written using SOLID principles)”. This gives the hiring manager an idea of how many projects you have written and also how you were involved in those projects.

 

Getting Noticed

 
This part is relatively easy for an IT contractor; I just posted my CV on JobSite, JobServe, ContractRecruit and Reed. After doing this I received no end of phone calls from recruiters. I would advise that posting your CV online too early, as this may have a detrimental effect – I shall have an update on this as I get nearer to actually going into contracting. Also, to appear more professional, I have been writing articles and doing open source projects that show off my skills, if you’re going to into software development, this seems like a good idea, as the head IT person may in fact look you and your work up.
 

Timing

 
There are two parts to this, firstly timing your actual leaving date, that it ties in with contracting seasons, and secondly, timing handing your notice in.

I’ve split this into two parts, because the minute you hand your notice in, the company you’re working for (if you’re any good) will want to negotiate to keep you. Although I had anticipated this, I didn’t anticipate how long it would actually take; so I would advice handing your notice in (the first time) at least a week before you want your notice period to start.

The next part if timing your leaving date to tie in with contracting seasons, the first few months of the year are when most of the contracting roles come out – people have come back from Christmas and want a new project started, and new budgets get issued. I personally tried to time my leaving date for the end of Jan, to get well into the contracting season… but with the negotiations with work, this has now become the start of February, which is frustrating.

That’s about it for this article. My next article will be posted on the 13th of Jan due to the Christmas period, and I also need to get one of my projects finished.

By

Diary of a web development contractor: Part 1

executive diary, a sleek cordless phone, and a cd-

Part 1: The Overview

With just two short months to go before I start my career as a contractor, I thought I’d start writing a “diary” for any other aspiring contractors to follow. I’ll be writing up articles on what steps to take, what I’ve done, what worked, what didn’t work etc… So any mistakes I’ve made can be avoided by other perms come interims.

First of all, how do you know you’re good enough to go into contracting? This is a difficult one to answer; for someone I know, he knew he was good enough when he knew more than a trainer hired to teach him and his colleagues. But for myself, because I’m a web developer, and development is such a huge thing (because of all languages, principles, methodologies, technologies etc…) it’s harder to know if you’re good enough. I’ve worked in a team of developers for a number of years, and I’ve worked with web technologies for a number of years too. I’ve learnt tons from my colleagues, and now I’m a senior coder that’s entrusted to write huge projects by myself – so I must be pretty good.

I would recommend working in a team before going into contracting – but ideally a team where at least one person is waaay smarter than you, or al least, as smart as you. Only when someone is smarter than you, do you realise how crap you actually are; I went through university, and two jobs thinking I was the dogs nuts, it wasn’t until I started my current job that I realised I wasn’t actually that great a coder.

Getting your skills in order

So first of all you’re going to need to pick a direction; desktop or web. Then you’ll need to choose your technology (MVC, Web Forms, Win Forms) and then your language (C#, PHP, VB.Net). Right now MVC/ASP.Net/C# is the current preferred flavour of technologies – an MS stack developer can earn £250-£450 per day depending on location and company.

Next, you’re gonna need to get shit hot with all the latest methodologies and principles. Right now Unit Tests, Test Driven Development and SOLID Principles are the preferred methodologies and principles. So you’ll need to seriously learn this stuff and write projects using these – ideally commercial projects. I’ve managed to learn about Unit Tests, TDD and SOLID by watching videos on YouTube, reading tons of tech articles, and writing a few projects.

Biting the bullet

Contract work moves fast; so you can’t have a one, two or three month notice period, because no company will wait that long. So you’ll need to bite the bullet and hand your notice in. For myself, I distributed my CV to recruiters this month, even though I still have two more months of notice to work, I though I’d get my name out there early – my phone hasn’t stopped ringing for the past two weeks.

I’ve also picked a prime time to hand my notice in, Jan/Feb is the best time for new contracts, so I made sure my notice period ends then. You’ll obviously need to save tons of money, as you can’t guarantee you’ll get work – you still need to interview like a normal job.

Preparing your CV

I’ll go into more detail on my next article, but you’ll want to re-write your CV and make it more project based, that was it is oriented towards contractual work. Whenever I was looking for work, I never used to get recruiters calling me about contractual work, now that I’ve re-worked my CV about 40% of the calls are about contractual work.

The legal stuff

Closer to the time you’ll need to set yourself up as a limited company; you can work under an umbrella, or as a sole trader, but a limited company is much better – I’ll cover more in a future article. Once setup, you’ll need to get yourself a good accountant, so that your yearly accounts can be completed, and you know how much to pay yourself (you can’t just take all the money that a contract pays you), and how much tax you’ll need to pay.

You’ll also need to sort out personal indemnity insurance, this will cover you against screw ups; this too will be covered in a future article.

That’s pretty much it

I’ll cover more as I go along, but these are the basic points you need to consider before attempting to move into the contracting sector.

By

Who’s fault is it anyway?

Information Leaks

These are worrying times we live in (in terms of IT), every week it seems that a major vulnerability has been found in a major application, or a web company has had it’s servers hacked. Yet, now more than ever companies are investing in internet-facing services – cloud services.

One has to wonder why it is that the UK government can’t trust a software company to write a secure electronic voting system, but yet major corporations entrusting their secrets to cloud based systems?

Cloud Technology – You’re cruisin’ for a bruisin’TM

Do you leave your possessions on your front lawn? Of course you don’t, but if you did, and your possessions were stolen, who’s fault is that? Yours for leaving them on the front lawn? Or the thief’s? (I know who the insurance company would blame) Yes, stealing is wrong, and the thief should be cause an punished, but on the other hand, you were foolish enough to leave them on your front lawn. Yet this is the reality of companies using cloud based services – instead of the hosting files on their own servers behind firewalls, and VPN only access, they are choosing to host their files on a publicly accessible system.

Now, I realise that there are Private Cloud systems, where the “cloud system” is hosted on the company’s servers, but then this would imply that the only possible attack point is the servers, but this simply isn’t true. In a cloud based system, the software is more vulnerable than the servers it runs on, this is because developers aren’t always aware of how a vulnerability can easily be introduced – take Cross Site Request Forgery for example, how obvious would it be to a developer, that a hacker can remotely run code on websites to mimic the actions of a user on a vulnerable website to do their bidding.

One of the more worrying aspects of cloud services, is not knowing where your files are. Consider a oil company; their data will be worth billions, they will have a file server to keep the confidential data on, they will have external backups with a company that can certify their security, old hardware will be certified by the recycling company that the hard drives have been destroyed, they will have security audits and policies in place so that employees cannot take data out of the building – they know where their data is at all times!

Now lets look at a company that uses a cloud based system – their files are hosted on a server… somewhere, their files are backed up by the cloud company and stored… somewhere, and when a file is deleted on a cloud based service, is it ever actually deleted? They have no idea where their data is!

Accessibility, or Vulnerability?

Once again, this down to the age old issue of “accessibility over security”. Users will want to make their lives easier, and their data more accessible at the expense of security; this is not deliberate, merely a consequence of how data is made more accessible. It is understandable how frustrating it is to have to connect to a VPN before you can access shared files, instead of having them readily accessible through a cloud service such as Google Docs or DropBox. VPN offers a secure login facility and traffic encryption, thus protecting the company’s data, it is possible to implement end-to-end encryption with Google Drive and DropBox, to protect the company’s data, but this then limits the ability to share files.

Encrypting files is a prime example of something that a general user wouldn’t think about, as it isn’t in their mind set, that a file on a cloud service could be stolen or viewed by someone outside the company.

I can see why IT will be seen as the “party poopers” when it comes to using new technologies, and making data more accessible. But bypassing IT when it comes to implementing any of this is just going to cause a world of hurt for the company, IT should see the growing trend of technologies and put policies in place, such as enforcing that encryption must be used when using cloud based services, or just putting a blanket ban on uploading highly confidential data (what data this covers would be defined in the company’s Data Security Policy).

The end is nigh

How long is it until we see a headline like “Vulnerability found in Google Docs”, one day this could be a reality, almost every week a new vulnerability if found in a major service. The greater worry is that an employee’s account is compromised (hackers are finding ever more inventive ways of achieving this), because cloud based services are public facing, this makes user accounts more vulnerable than ones on Active Directory network behind monitoring and firewalls.

In Google Doc’s Terms and Conditions, it states quite clearly that they not liable for any damages, and I would assume that other cloud services also say pretty much the same thing. So if they do get hacked and your data is stolen, the company will just say “oops”, and fix the vulnerability (if possible). Your rivals now have access to your data, and there is virtually squat you can do about it.

A breach of security in a cloud based service is almost certain to happen, no matter how much you train your developers in security, and no matter how much the company sends on Pen Testing, something will always slip through the cracks. Microsoft spend millions on Pen Testing windows, and yet a major vulnerability has just been discovered, after being implement 19 years ago.

Conclusion

It is pretty clear that hosting confidential data on a cloud service will eventually bite you in the a**, as Twitter found out 5 years ago, other cloud based file services have already discovered security vulnerabilities.

One thing is for certain; cloud based services are here to stay, so IT departments will need to find a way to allow users to access them, just as they are starting to have to allow users to bring their own devices to work.

By

Creating a settings class in C#

java source code

My current project the Secure Password Repository required a class that stores the global settings for the entire system. I wanted the class to be; updatable by the system’s administrator(s), load from a settings file upon instantiation, and also save updated settings back to the file. Also, I only wanted there to be one instance of the class across the entire application, and it needed to be fast – no loading from the disk each time setting values were requested.

Gettings Started

The settings need to initially load from disk, normally, one would write something that reads an XML file and then stores the values into the class members, but in this case it makes more sense to make the class serializable. By making the class serlizable, we can just use the XmlSerlizer to values directly from the value into the class members, and when write them back to disk after updates.

To make the class serilizable, the [Serializable] attribute just needs to be added:

[Serializable]
public sealed class <classname> {
}

A static instance of a serliazer is then needed:

private static readonly XmlSerializer serial = new XmlSerializer(typeof(<classname>));

This will allow the class member values to be written to disk like so:

using (StreamWriter sw = new StreamWriter(filename))
{
     serial.Serialize(sw, instance);
}

The values can be read from disk like so:

instance = (<classname>)serial.Deserialize(sr);

The members aren’t declared in any special way, however, numeric values have issues being read from disk as everything is stored in text format:

public string LogoImage { get; set; }
public string PBKDF2IterationCount { get; set; } //NUMERIC VALUE
public bool BroadcastPasswordPositionChange { get; set; } //BOOLEANS LOAD OK

Thread safe

When an instance of the class is first requested, it will need to load the values from disk (or cache), this can lead to issues if two users request the instance at the same time. So the class needs to “lock” itself, load the values, then “unlock” itself. Whilst being locked any other requests need to go into a queue until the class has finished initialising. Luckily .Net has a lock method for this:

//check if an instance of this class is already in cache
if (MemoryCache.Default.Get(filename) != null)
{
     return (ApplicationSettings)MemoryCache.Default.Get(filename);
}

//there isnt an instance in cache, so we need to create one
//lock this thread, so that only one thread creates an instance
lock (thisLock)
{
     //do all the loading stuff here
}

The object “thisLock” is anything more than an object declared specifically for the locking:

private static Object thisLock = new Object();

First load

Once the web application has been installed, ideally there shouldn’t be a settings file supplied, because (a) you’ll want the default values to be loaded and (b) that last thing a developer/company would want is a developer’s test values being supplied. So the first thing the class needs to do is, check if the settings file exists, if not, then set default values and create the file.


//if the file does not exist
if (!File.Exists(Path.Combine(HttpRuntime.AppDomainAppPath, "system-config.xml")))
{
     //create file
}
//use a streamreader to read the contents of a file from disk
using (StreamReader sr = new StreamReader(filename))
{
     //serialize the object from disk and create the singleton object
     instance = (ApplicationSettings)serial.Deserialize(sr);
}

That pretty much covers the important parts of creating a settings class. Below is a full listing of the setting class code, this is also available for download from my secure password repository project on GitHub: https://github.com/thatcoderguy/Secure-Password-Repository

Full code listing


[Serializable]
public sealed partial class ApplicationSettings
{

     //create a new xml serializer to serlize this onject from disk
     private static readonly XmlSerializer serial = new XmlSerializer(typeof(ApplicationSettings));

     //an instance of this class - thus this is a Singleton
     private static volatile ApplicationSettings instance;

     //for locking
     private static Object thisLock = new Object();

     ///

     /// Persist changes to disk
     ///

     private static void Save()
     {
     //serialize this object back to disk
     string filename = Path.Combine(HttpRuntime.AppDomainAppPath, "system-config.xml");
     using (StreamWriter sw = new StreamWriter(filename))
     {
          serial.Serialize(sw, instance);
     }

     //make sure the changed values are copied back into cache
     MemoryCache.Default.Set(filename, instance, new CacheItemPolicy() {
                                                  AbsoluteExpiration = MemoryCache.InfiniteAbsoluteExpiration,
                                                  SlidingExpiration = MemoryCache.NoSlidingExpiration,
                                                  Priority = CacheItemPriority.Default });
     }

     ///

     /// Reset application settings to default values
     ///

     public static void ResetAppSettings()
     {
     Default.LogoImage = "logo.png";
     Default.SMTPServerAddress = "localhost";
     Default.SMTPServerUsername = string.Empty;
     Default.SMTPServerPassword = string.Empty;
     Default.SystemInitilisationVector = EncryptionAndHashing.Generate_Random_ReadableString(16);
     Default.SystemSalt = EncryptionAndHashing.Generate_Random_ReadableString(32);
     Default.SCryptHashCost = "262144";
     Default.PBKDF2IterationCount = "1000";
     Default.AdminsHaveAccessToAllPasswords = true;
     Default.RoleAllowAddCategories = "User";
     Default.RoleAllowDeleteCategories = "Administrator";
     Default.RoleAllowEditCategories = "Administrator";
     Default.RoleAllowAddPasswords = "User";
     Default.SMTPEmailAddress = "securepasswordrepository@local";
     Default.BroadcastCategoryPositionChange = false;
     Default.BroadcastPasswordPositionChange = false;

     Save();
     }

     ///

     /// Update application settings
     ///

     ///      public void UpdateSettings(SystemSettingViewModel newSettingsModel)
     {
     //TO DO: extra validation checks at some point - as these are important values
     Default.LogoImage = newSettingsModel.LogoImage;
     Default.PBKDF2IterationCount = newSettingsModel.PBKDF2IterationCount;
     Default.SCryptHashCost = newSettingsModel.SCryptHashCost;
     Default.SMTPEmailAddress = newSettingsModel.SMTPEmailAddress;
     Default.SMTPServerAddress = newSettingsModel.SMTPServerAddress;
     Default.SMTPServerPassword = newSettingsModel.SMTPServerPassword;
     Default.SMTPServerUsername = newSettingsModel.SMTPServerUsername;
     Default.RoleAllowAddCategories = newSettingsModel.RoleAllowAddCategories.Name;
     Default.RoleAllowAddPasswords = newSettingsModel.RoleAllowAddPasswords.Name;
     Default.RoleAllowDeleteCategories = newSettingsModel.RoleAllowDeleteCategories.Name;
     Default.RoleAllowEditCategories = newSettingsModel.RoleAllowEditCategories.Name;
     Default.AdminsHaveAccessToAllPasswords = newSettingsModel.AdminsHaveAccessToAllPasswords;
     Default.BroadcastCategoryPositionChange = newSettingsModel.BroadcastCategoryPositionChange;
     Default.BroadcastPasswordPositionChange = newSettingsModel.BroadcastPasswordPositionChange;

     Save();
     }

     ///

     /// Returns default instance of this class
     ///

     public static ApplicationSettings Default
     {
     get
     {

          //get cache and the file name
          string filename = Path.Combine(HttpRuntime.AppDomainAppPath, "system-config.xml");

          //check if an instance of this class is already in cache
          if (MemoryCache.Default.Get(filename) != null)
          {
          return (ApplicationSettings)MemoryCache.Default.Get(filename);
          }

          //there isnt an instance in cache, so we need to create one
          //lock this thread, so that only one thread creates an insrance
          lock (thisLock)
          {

          //the thread has been unlocked

          //check that the object hasnt already been put into cache by another thread
          if (MemoryCache.Default.Get(filename) != null)
          {
               return (ApplicationSettings)MemoryCache.Default.Get(filename);
          }

          //if the file does not exist
          if (!File.Exists(Path.Combine(HttpRuntime.AppDomainAppPath, "system-config.xml")))
          {
               //create a new instance of this class
               instance = new ApplicationSettings();

               //insert the object into cache - with no expiration (we want this to be persistant in memory)
               MemoryCache.Default.Set(filename, instance, new CacheItemPolicy() {
                                                       AbsoluteExpiration = MemoryCache.InfiniteAbsoluteExpiration,
                                                       SlidingExpiration = MemoryCache.NoSlidingExpiration,
                                                       Priority = CacheItemPriority.Default });

               //save to disk
               Save();

               //set the default values
               ResetAppSettings();

               //return the newly created object
               return (ApplicationSettings)MemoryCache.Default.Get(filename);

          }

          //use a streamreader to read the contents of a file from disk
          using (StreamReader sr = new StreamReader(filename))
          {

               //serialize the object from disk and create the singleton object
               instance = (ApplicationSettings)serial.Deserialize(sr);

               //insert the object into cache - with no expiration (we want this to be persistant in memory)
               MemoryCache.Default.Set(filename, instance, new CacheItemPolicy() {
                                                       AbsoluteExpiration = MemoryCache.InfiniteAbsoluteExpiration,
                                                       SlidingExpiration = MemoryCache.NoSlidingExpiration,
                                                       Priority = CacheItemPriority.Default });

               //return the newly created object
               return (ApplicationSettings)MemoryCache.Default.Get(filename);

          }

          }

     }
     }

     //class properties (settings for this app)
     public string LogoImage { get; set; }
     public string SMTPServerAddress { get; set; }
     public string SMTPEmailAddress { get; set; }
     public string SMTPServerUsername { get; set; }
     public string SMTPServerPassword { get; set; }
     public string SystemSalt { get; set; }
     public string SystemInitilisationVector { get; set; }
     public string SCryptHashCost { get; set; }
     public string PBKDF2IterationCount { get; set; }
     public string RoleAllowEditCategories { get; set; }
     public string RoleAllowDeleteCategories { get; set; }
     public string RoleAllowAddCategories { get; set; }
     public string RoleAllowAddPasswords { get; set; }
     public bool AdminsHaveAccessToAllPasswords { get; set; }
     public bool BroadcastCategoryPositionChange { get; set; }
     public bool BroadcastPasswordPositionChange { get; set; }

}

By

Secure Password Repository Released

My Secure password repository project has finally been released.

You download the source code from here: https://github.com/thatcoderguy/Secure-Password-Repository

Some of the technologies being used are:

  • Signal R
  • MVC
  • Entity Framework
  • LINQ 2 SQL
  • C#
  • AutoMapper
  • Web Activator
  • Identity 2.0

By

Building a TO DO list in GitHub

To Do list with check marks Using GitHub as an open source project version control system, is the best decision I ever made, compared to of version control systems, it is so much more manageable.

However, one feature I think is missing, is a TO DO list function – if you’re managing my open project on GitHub with several collaborators, it makes sense that you’d want a central TO DO list, as each member can be assigned a feature (or assign themselves), and the TO DO items can go into a Milestone.

I did a quick Google for how to build a TO DO list in GitHub, and I came across one blog; http://lifehacker.com/why-a-github-gist-is-my-favorite-to-do-list-1493063613 which seemed great when I first implemented it, but now I realise just how lame it is, and how much better my current solution is.

Building a TO DO List – The lame way

So this method came directly from the blog post above; first create a new file called todo.md:

create new file

Each item is then added in with the text “- [] Item Text” e.g. “- [] 1. First Item”

todolist
preview1

Saving the file then displays the TO DO list in the format above, which actually looks pretty nice with the check boxes. The bit that makes this method lame, is you can’t tick the boxes, instead you have to put an X inside the [], so that GitHub knows to show it as a tick:

completed
preview2

The method above creates a nice looking TO DO list, but to it isn’t very function, plus it will be a pain in the *rse to assign developers to tasks and group tasks into milestones. The next method is much better, you can; add notes to the task, assign developers, associate milestones, tick items off as complete, attach images and add notifications.
 

Building a TO DO List – The awesome way

 
First go into Issues, then into Labels:

issues2

Once inside labels, click on “New Label”, in the text box, type “TO DO”, assign a colour (ideally something that stands out), then click on “Create Label”:

newlabel

Now that you have a label created, you can now start to create TO DO items, and assign them. Click on “Issues”, click on “New Issue”. Once on the creation form, enter the name of the item, click on the “TO DO” item in the labels, and optionally assign a developer and milestone to the item.

createissue

The item will then display on a list screen amongst bugs, and other issues – but the screen can be filtered for TO DO items only:

LIST

As you can see this is a much better way of creating a TO DO list item, than the first method above. It does kind of suck that your TO DO items will come under “issues”, it would be so much better if there were a TO DO list area.

By

E-Commerce shoplifting (or how not to build a payment gateway)

I was looking at how some payment gateways integrate with websites, because I wanted to build my own payment gateway protocol. And whilst doing this, I came across quite a serious flaw in a certain payment gateway’s security protocol. I won’t name the payment gateway, but the vulnerability only exists in the old version of their protocol… however, there are a significant number of sites still using this version.

Disclaimer: The information in this blog article is for educational purposes only. If you forge a callback to an website, as I will show you in this article, you are committing fraud. This is a felony, and I will not accept any responsibility for someone actually trying this out on a website.

The integration protocol for the payment gateway in question is pretty simple, and looks like this:

protocol

This is a standard protocol when it comes to payment gateways; send data to the gateway (hashed or plaintext with a shared secret via HTTPS), gateway takes payment, gateway sends data back to site (compare hash or compare shared secret), site processes order. The payment gateway (that will not be named) has done things slightly differently to the others, first they have asked for the data to be encrypted with XOR encryption (most other gateways use hashing), and secondly, they do not use a HMAC to verify the contents of messages. These two combined mean that the gateway’s protocol is flawed and anyone who knows what they are doing can fake a callback to the website, and have their order processed without any payment.
 

How can we exploit this flaw?

 
First I’ll discuss what XOR is, then how XOR encryption works, so that discussing how to crack the encryption will be more understandable.

A CPU is built up on a number of logic gates, such as: AND, OR, NAND, XOR. These gates accept 2 inputs, and give out one input, and the output depends on the values from the two input signals. For example, if we take an AND gate, this is the logic table that is derived from all the possible input options:

Input 1: On Input 1: Off
Input 2: Off Output: Off Output: Off
Input 2: On Output: On Output: Off

Only when both inputs are On, is the output On.

Now lets look at the OR logic table:

Input 1: On Input 1: Off
Input 2: Off Output: On Output: Off
Input 2: On Output: On Output: On

When Input 1 is On or Input 2 is On, then the output is On.

This brings us to the XOR logic table:

Input 1: On Input 1: Off
Input 2: Off Output: On Output: Off
Input 2: On Output: Off Output: On

Only when Input 1 is On or Input 2 is On, is the output On.

So how does this work with encryption?

Let’s take a single plaintext message char: B and also a single encryption key char: a

B has an ASCII value of 66, converted to binary this is 01000010
a has an ASCII value of 97, converted to binary this is 01100001

XOR encryption works by taking a binary digit from the message, and a binary digit from the key and XORing them. This is then repeated for all the digits. e.g.

B 01000010
a 01100001
Output 00100011

00100011 converted to decimal is 35, looking at an ASCII table, char 35 is the # symbol.
So B XOR encrypted with a gives an output of #.

The XOR encryption is repeated for each char in the plaintext message, if the message is longer than the key, then the key must be repeated, which means repeating chars in the encrypted string, and this is a weakness in XOR encryption. Another interesting thing about XOR encryption (and this will allow us to break the encryption of a real system later) is that you can derive the key, if you know the plaintext:

# 00100011
B 01000010
Output 01100001

Convert 00100011 to decimal gives you 97, which is in ASCII is a. As you can see; encrypting the encrypted text with the plaintext will give you the encryption key originally used.

Anyone familiar with the payment gateway’s protocol (or has read the developer’s guide on the website) will know that the encryption key is 16 bytes long, and that the encryption key will only be in the A-Za-z0-9 range of chars. This means that there are 62^16 (or 47,672,401,706,823,533,450,263,330,816) possible encryption keys, which will take a loooooong time to brute force… as shown above, there is a shortcut.
 

Cracking XOR encryption

 
Anyone familiar with the payment gateway’s protocol will know that the data being sent to the server looks something like this: GatewayOrder=33525&ordertotal=100.33&returnurl=http://website.com&billingname=… etc. All of the chars in the data being sent will be within the printable range.

This means that the first 13 chars of the string being sent to the server will always be: GatewayOrder=, which is then followed by the order number. So if we encrypt the encrypted message with the text GatewayOrder=, this will give us the first 13 chars of the encryption key. The last 3 chars will be the first 3 numbers of the order number, this is usually sent in the query string data in plaintext, but on the off chance it isn’t, you still only have 62^3 (or 238,328) keys to break – I’ll show you how to further reduce the key range later.

Let’s look at a working example; firstly to grab the encrypted message, we can just process an order on any website (using the payment gateway), the order details get POSTed to the payment gateyway, so we can just use the standard developer tools built into Internet Explorer:

realmessage3

Note: I have hidden some fields and changed some values so that the payment gateway cannot be identified, and also to make the demonstration easier.

You can see the encrypted message being submitted to the gateway is: CSBAIDI1En89NQoDe0QDZHx0Eio3MA5COz4bECpKAWF+bwd2YyYORDojAQQ0Gw05OjVEf2p7HFUtIgYFI1lTPiNnViwpOAJeKD8OHCNKXjAjJBInLDgHWSE2DhUiBVUiPXAJNio5DlErNR0UNQQWMyctWCwrMwpUKyMKAjVFDTAgLkAtICYHWSE0SQEpBEQyISVReCEyWAR7YwsX
The text is Base64 encoded, so the text will need decoding, which is trivial: byte[] encrypted = Convert.FromBase64String(“CSBAIDI1En89NQoDe0QDZHx0Eio3MA5COz4bECpKAWF+bwd2YyYORDojAQQ0Gw05OjVEf2p7HFUtIgYFI1lTPiNnViwpOAJeKD8OHCNKXjAjJBInLDgHWSE2DhUiBVUiPXAJNio5DlErNR0UNQQWMyctWCwrMwpUKyMKAjVFDTAgLkAtICYHWSE0SQEpBEQyISVReCEyWAR7YwsX”); this produces the following text: encrypted3 I’ve had to display the text as an image, because most of the chars aren’t in the printable range.

Now, to get the encryption key, that was used to encrypt that data, we just need to encrypt the first 16 bytes of the encrypted string with GatewayOrder=335. As mentioned previously the first 13 chars being sent to the server will always be GatewayOrder= and the next 3 chars will be part of order id – which you can see highlighted in the image above.

Here’s the complete code listing that will reveal the encryption Key:


     static void Main(string[] args)
     {

          byte[] encrypted = Convert.FromBase64String("CSBAIDI1En89NQoDe0QDZHx0Eio3MA5COz4bECpKAWF+bwd2YyYORDojAQQ0Gw05OjVEf2p7HFUtIgYFI1lTPiNnViwpOAJeKD8OHCNKXjAjJBInLDgHWSE2DhUiBVUiPXAJNio5DlErNR0UNQQWMyctWCwrMwpUKyMKAjVFDTAgLkAtICYHWSE0SQEpBEQyISVReCEyWAR7YwsX"); //store the encryted text
          string plainttext = "GatewayOrder=335"; //store the first 16 bytes of the text

          Console.WriteLine(System.Text.Encoding.Default.GetString(System.Text.Encoding.Default.GetBytes(calcXor(plainttext, System.Text.Encoding.Default.GetString(encrypted)))));
          Console.Read();
     }

     public static char[] calcXor(string a, string b)
     {
          char[] charAArray = a.ToCharArray();
          char[] charBArray = b.ToCharArray();
          char[] result = new char[a.Length];
          int count = 0;

          for (int i = 0; i < a.Length; i++)           {           if (count > b.Length-1)
               count = 0;

          result[i] = (char)(charAArray[i] ^ charBArray[count]);
          count++;
          }

          return result;
     }

And this is the output:

programoutput

Which is the encryption key. Great, so we have the encryption key now we can forge a callback, and fool the website into thinking we’ve paid for our order. But before we do that, I just want to demonstrate how easy it will be to crack the XOR when the plaintext string isn’t fully known.

The reduce the key range into a size that is crackable, you just need to XOR all printable chars (A-Za-z0-9) with each char in the encrypted text, and see if the result is a char that is accepted in a URL (ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789&=%) below is the complete source code to do this:


     string encrypted = "CSBAIDI1En89NQoDe0QDZHx0Eio3MA5COz4bECpKAWF+bwd2YyYORDojAQQ0Gw05OjVEf2p7HFUtIgYFI1lTPiNnViwpOAJeKD8OHCNKXjAjJBInLDgHWSE2DhUiBVUiPXAJNio5DlErNR0UNQQWMyctWCwrMwpUKyMKAjVFDTAgLkAtICYHWSE0SQEpBEQyISVReCEyWAR7YwsX"; //encrypted base64 string
     encrypted = System.Text.Encoding.Default.GetString(Convert.FromBase64String(encrypted));

     string keychars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"; //possible chars that can make up the encryption key

     for (int i = 0; i < 16; i++)      {      StringBuilder sb = new StringBuilder();      for (int n = 0; n < keychars.Length; n++)      {           if ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789&=%".Contains((char)(keychars[n] ^ encrypted[i])))           {           sb.Append(keychars[n]);           }      }      Console.WriteLine(sb.ToString()); //output possible chars      Console.WriteLine(sb.Length); //output number of possible chars      }

The output of this program is:

encrypted5

Admittedly this still produces quite a large number of possible keys, but if you know part of the plaintext (as we do), then cracking the remaining chars becomes easier. If we modify this line: for (int i = 0; i < 16; i++) to read: for (int i = 13; i < 16; i++) as we already know the first 13 chars, and just need to find the final 3. This outputs the following:

encrypted6

Which is only is only 20,216 possible combinations to try.
 

How to fake the callback to the server

 
Now that we have the encryption key, we just need to encrypt a response and send it to the server, so that the website thinks we paid for the order.

The data being sent to the gateway included the URL that the gateway should send the response to. This is done via a simple redirect, rather than via a form POST (as most other gateways do), so really all we need to do is generate the URL with the query string that the gateway would send, and just visit it in our browser.

The gateway sends the following query string after a successful purchase: [URL]&encrypteddata=[encrypted base64 data]. The website will then take the encrypteddata query string value, decrypt it, and see what the payment gateway message is.

According to the documentation, the data sent back from the payment gateway should be: GatewayOrder=[order number]&TransactionID=[transaction id]&Payment=[payment status]&PaymentMessage=[Any message from the gateway]&OrderTotal=[amound paid]

So for a successful payment, the message would look something like this (which the developer's guide will show):

GatewayOrder=33525&TransactionID=768768-24424223&Payment=OK&PaymentMessage=Payment Successful&OrderTotal=100.33

So all we have to do is encrypt this with the encryption key, and submit it to the site. The code listing of the first program above can be modified to encrypt the full text, which gives this output: CSBAIDI1En89NQoDe0QDZHx0EhE3NQVDLjIbGCkZeRVzdgJ9cmJTHX1lW0NyRQJiaBFVPCgxBURyHiRXFhZJPCsvQAggJxhRKDRSIScOXTQgNRQWMDcIVTwiCQQqUX8jKiRGESogClxyYF9BaEQD

After decrypting the message sent to the payment gateway, you will see the returnurl parameter, which may look something like: http://www.websitecom.com/process.asp?websitename=MY Website. To forge the callback all you need to do is append the encrypted data like so: http://www.websitecom.com/process.asp?websitename=MY Website&encrypteddata=CSBAIDI1En89NQoDe0QDZHx0EhE3NQVDLjIbGCkZeRVzdgJ9cmJTHX1lW0NyRQJiaBFVPCgxBURyHiRXFhZJPCsvQAggJxhRKDRSIScOXTQgNRQWMDcIVTwiCQQqUX8jKiRGESogClxyYF9BaEQD. You can then just paste this into your address bar, and press enter.

That's it. You should now be redirected to the payment success screen of the website. This all worryingly too simple, and it's a wonder why crackers don't use this all the time.
 

Preventing this attack

 
As the payment gateway cannot possibly replace their encryption scheme (as this would probably cost millions), there are few options developers have to prevent this attack. You can check the referrer, to see if the request came from the gateway, but referrers can easily be forged. Another option would be to use the latest version of their protocol which does implement a HMAC, and can therefore be used to validate the contents of the callback. A final option would be to move from form based to direct server based, so that the client never sees the data being passed back an forth.
 

Conclusion

 
It is amazing that huge companies can write software riddled with flaws like this, there will eventually have to be a major wake up call, due to the recent news of websites being hacked. There are a number of simpler payment gateways that have flaws that are event easier to exploit, which I may cover in future articles.