Hallway Talks: Threat Levels

My company has started holding something we call an un-conferance... basically it is an open space where we can give less formal conference style talks (see https://en.wikipedia.org/wiki/Unconference) but more on that later.

Anyway a friend of mine gave an interesting talk at the security un-conference today on actually talking about security that brought to mind an interesting conversation I had with a group of pen testers after the Belfast Bsides event this last year.

He spoke about communicating the reasons for security operations to happen with the customers/consumers it will effects and likened it to having airport security being one of the worst experiences when travelling to the point where taking a flight used to be a simple and great experience people would have looked forward to having become a tiresome pain- waiting in lines to be checked over, and even having your favorite soft drink taken off you. Most people will tell you the security in airports now is one of the worst parts of travelling a classic security vs convenience situation is created.

So if we tell people why we are doing the checks will it make people more receptive to adopting the change? Or another side would simply having to explain the threats to the people affected make it very obvious that even if that threat is severe the risk is low enough that it doesn’t warrant the inconvenience?

This was a core point of his talk, we always hear about threat but rarely hear about risk and I remembered this conversation I had in the post-conference celebration with a group of pen-testers in about automated vehicles. One of the days talks focused on how manufacturers of these vehicles so far were not putting an emphasis on security and how you an easily break into them, this also happened with the hackers ale to take control remotely of a partially automated range rover and force it to stop, so a fairly hot topic. Now regardless of how you feel about the possible threat of this happening I didn’t and still don’t believe it would be a reason to not use automated vehicles. My argument for this is that yes the threat is severe but the risk of them getting hacked is trivial.  

Now before you jump on the “your insane, it’s terrifying” … yes the risk is a terrifying prospect but let’s assume the security on the car systems is essentially absent and just pretend that a there is no security whatsoever and you can log into its open hot spot log into a shell and type a command to have the car go or stop. Even then there’s a barrier to entry, the person attempting would need a level of IT skills, a background knowledge of the software its running on and equipment with the ability to extend the range of there connection long enough so the vehicle doesn’t leave it. Of course the level of fear of the threat will never allow such minimal precautions but remember the people who managed to get the car in the news had all that equipment experience and training. They proved it possible not likely, you could achieve the same out come with a pair of snips and an idea what the brake line looks like. I believe that method of attack would be only used for assassination or terror. But the mindset of the pen tester is different, they see attacks everywhere they were insistent kids would find ways of doing this and just murder people for the fun of it and even after there was convictions of what is clearly pre-meditated murder and the security brought up to high levels you would still get the odd person who would sit there and figure out how to brake it.

It’s hard to argue with what is possible.  Even with the more security you put on something, in this example an automated car, the higher the barrier to entry of attack is… but it will always be a possibility. I think with the level of fear in automated cars the manufacturers will put effort into securing them especially if there liable for negligence leaving that risk trivial but please tell me what you think?

There is an important take away from this however.

•          When you’re focused on the possibilities and not likelihood it’s easy to overestimate the level of mitigation required.
•          Security needs to be invisible, effective and at least appear seamless to the end customer.
•          When explaining risks to someone who doesn’t agree, pushing that risk possibility can make your argument sound even more ridiculous or far-fetched (this is where people get annoyed with airport security).

Data Driven Testing with JUnit Theories and Parametrization

Theories are part of the J Unit test suite and run the same way that parametrization tests work. The function of a theory is to test a large amount of data with a relatively small number of tests.

The main difference, as I see it, between theories and parametrized tests is that with a theory you are testing essentially data that has been generated prior to the test and parametrized tests are better suited to running a large amount of variables through tests to validate them. You see this when you run it, a parametrized test will generate a test for every iteration. This makes it easy to validate a wide range of data. Where a Theory will generate a single test and fail if all of the data is not correct, so its less of a validation more just ensuring what's expected is there.

Theories take data from a parameter marked as a @Datapoint. and uses all the data it in the following methods, when 2 or more methods are used along with data points it works in the same way a 2D array would with a for each loop iterating one over the next until it has used every possible combination of the data points.

Theory's are handy over parametrised tests in that they are less "busy", that is... they are short and easy to read/understand and give a clear (concise) output. Parameters require a little more work and a little more effort to understand but are very limited in there use cases.

Walk through

Lets take a step back from what its for and see how we can use theories and parametrized tests.

Theories

 First declare that the class will be using Theories above the class declaration:

@RunWith(Theories.class)
public class TheoriesClass {

This lets our class know its to run using the Theories test runner.

  Next inside the Class declare what data you will be rerunning through the method this is done using the annotation @DataPoints/@DataPoint (Datapoints not available in earlier versions)

The line following can either be a straight forward declaration of

 @DataPoints
public DataItem[] dataToBeTested(){
return stringArrayFromFile();
}

once the data has been declared you can create the method that the data is run through, We can do this by anointing the test with

@Theory
public void Theory_Test(DataItem datapoint){
assertThat(datapoint, hasItem('user'));
}

and that's it, fairly straightforward way to assert that an object or variable contains something that it must. I find one big differences in using this method is that you don't assert that something is true or false or works in a certain way. The theory will assert a object has a matcher in it or certain value easily enough but not process that data once it has been passed to the theory...that's what a parametrized test is for.

Observations with Theories...

...! if you put in 2 or more variables into the test it iterates over them recursively

e.g..

@Theory
public void test(int first, int second){
System.out.println( first * second+", ");
}

with the data points [1,2,3] will show us an output like : 1, 2, 3, 2, 4, 6, 3, 6, 9

..! Theories should be kept to assertions about data that is supposed to relate, trying to do extra processing isn't what its for and will result in frustrations. i.e. modify data to make it readable as use asserts such as assumeThat()

 Parametrized Testing

Very similar to theories ( although more of a precursor to theories)

Parametrized tests have more straight forward use cases

To start same as theories we declare the testrunner

           @RunWith(Parameterized.class)

            public class ParameterClass {

 Next  the Parametrized tests use global variables and a constructor so add a something similar to the snippet below

private String inputNumber;  //globals
private String expectedResult;     
public ParameterClass (String inputNumber,  String expectedResult) {    // Constructor 
this.inputNumber = inputNumber;        
this.expectedResult = expectedResult;     
}

This is one of the big differences with Parametrised tests they use constructors and global in order to achieve repeating the same test over and over.

The variables in the constructors are defined by the @Parameter method/declaration.

In this example you will notice 2 inputs for the tests, adding more inputs is a little trickier using this type of test, since each variable must have a object to bind to, unlike above it wont just replicate and iterate through each scenario.

@Parameters(name = "Country code test:  {0} Translates to {1}")

public static Collection<Object[]> Setup(){
 String data = Utils.readInCSV();
   String lines[] = data.split(":S:");
        Object codes[][] = new Object[length][2];
         for (int i = 0; i < lines.length; i++) {
  String temp[] = lines[i].split(",");
  codes[i][0] = temp[0];
  codes[i][1] = temp[2];
 }
     return Arrays.asList(codes);
 }

From that snippet we can see a few things. First notice the additional tag for the name, it i an optional part but allows you to name each of the tests and is very useful when running a lot of data or putting the test into a build where there may be failed components. If you don't add the name annotation they will just be called by the number of the test which can cause pain debugging later.

Next notice how the object is returned in the return statement so it can be bound to the global variables. the Array has 2 objects in it and they are bound in the order of the class constructor. So for the example the object at Array[1] will be inputNumber and Array[2] will be expectedResult. 

Finally we get too the class itself, as you can see this one is the same as above except since it is created for each iteration of the test, we use the current global variables to test against and don't pass anything into the test itself.

@Test
public void occupationMappingTest() {    
     Pattern p = Pattern.compile("[a-zA-Z]{2,3}");
 Matcher m = p.matcher(output);
 assertTrue(m.matches());
}

and there you have it! 

using these methods you can validate large amounts of data with relative ease.

Feel free to check out my example Hosted by Google until i get a chance to upload it to github, the code snippets are somewhat psudo for explanation purposes

https://drive.google.com/open?id=0BzYAUrAfCDXIbFoxdHJjR3RsY2M