Friday, January 18, 2019

AI and the Internet: Sometimes it feels like the 1990s again

Over the past few months I have been (as well as editing the thesis) looking at the world of Artificial Intelligence developments, mainly in relating to working it into a module on an undergraduate BA course called Technology and National Security. (That's the name of the module which is 13 two hour lectures and 13 two hour tutorials, covering everything from the nature of war, through military ethics, to robotics, drones and automated weapons, with a quick detour through cyber-security and global security governance.  Very interesting if intensely depressing subject matter. Much more on this module in posts over the next few months as I finish up the materials).

Anyway, the over-riding feeling I was getting when looking at the state of AI developments was that it was just like the commercialisation of the Internet in the 1990s.    In brief:

1. Nobody is too sure how it is going to play out in the long term. 

Just like the commercial Internet in the 1990s there is a huge question over when (and whether) AI - as in General AI - will really happen.  Nobody was sure of online shopping (I know, it seems incredible now) - there was no guarantee that Amazon would ever make it, ('nothing could replace the smell of a book shop' and all that good stuff), delivery was difficult, clothes would never be sold online because you had to try them on - I could go on.  Debates about AI feel very familiar,

2. Hybrid (or partial) solutions as a temporary fix. 

The old Internet days saw lots of partial solutions - in fact there was so much concern about credit cards online at one point that one model proposed was that research would be done online and ordering would be done either in a shop or by telephone (ironically I have started taking photos of items in shops and then looking for them online - so the model has completely inverted).  AI is full of partial solutions - machine learning, automation etc are all really partial AI - as are the narrow AI implementations that get all the publicity - very good at one thing.

3. Huge enthusiasm is the order of the day.

There is huge enthusiasm for the task of achieving AI - any type of AI.  Just like working on early Internet services!  We were changing the world back in the 1990s, just like  the AI pioneers believe they are today.

4, Making it work is the priority

Another similarity is the emphasis on just getting it to work.  The technical problems that dominate AI are (in some ways) the direct descendants of getting a multi vendor networking environment to communicate with itself!  The problems are inherently technical, and the approach is to find solutions to specific problems.

And so what?

Well, if you look at the Internet of 2019 it sure as hell isn't the Internet we envisaged in 1990.  It's a basket case of unconsidered issues, abuse, misuse, and the fact that design considerations never survive first contact with a user.

There was a time when the most subversive thing on the Internet was a dancing hamster (I exaggerate to make a point), but now the Internet is now an enabler of organised crime, child abuse, people trafficking, the sale of drugs and weapons, psychological warfare, bullying, disinformation, state influence operations, invasions of privacy, collection of personal data, and poorly written blogs (so many poorly written blogs).

And my point.  There is a lot that AI development can learn from the Internet.  Given the inherent dangers of AI (humanity's last invention, huge surpluses of paperclips and all that good stuff) then the lets make it work approach is not enough.   There has to be consideration of the wider issues - one of which is that some of the worst abuses of the promise of the Internet are at the hands of the State.

The evidence of the Internet would suggest that trusting nation states to provide the governance necessary for AI would be a mistake.


Non-Academic Publishing

As part of the PhD process that is now (thank heavens) rapidly approaching its end, there have been several discussions suggesting that publishing some papers in 'respected academic journals' would be 'a good thing'.   There are a number of chapters in the PhD that could be carved out and turned into stand alone papers, but I have to be honest, after nearly four years, I am more likely to tear them out and burn them. 

I digress.

Having spent four years reading academic journals, I'm  not so sure about the value of contributing to them.  If I want to have a pointless debate about issues of definition I can do that when taking the kids to school. (This morning's starter for ten:  "Now that my son is 18 and technically an adult,does this mean my daughter is an only child?"  Son's view is no, daughter's view - well you can probably guess).

I digress again. 

The issue for me (and if you are one of my 12 regular twitter followers you will know this) is that academic publishing is basically a scam.   Buying journal access is a huge cost for Universities, and buying academic books that are required by students and academics and normally bought by the University libraries is also a huge cost. 

This severely limits the availability of the information - which actually in some cases may not be a bad thing - but defeats what should be the point of academic publishing - the sharing of knowledge.

Certainly in my subject area the far more valuable information is from commercial organisations who realise the value of sharing their expertise with a wider community and do so for free in most cases. With academic publishers if you write something then you have to pay to have it made freely available.

Given the potential outlets for information and ideas, and the fact that any 'editorial board' could never match the critical capability of my 12 regular followers I shall be using this blog and then, probably annoyingly cross posting to Twitter and LinkedIn until I get blocked.

I am not by nature an academic.  I like to see things get done rather than see things get talked about.  I spent years working in organisations where the biggest limiting factor was 'analysis paralysis' as more and more individuals who frankly should have been working in academia spent their time discussing how many angels could dance on the head of a pin - or whatever the telecoms equivalent would be.

I do not enjoy writing in an 'academic style'.  I like humour, digressions, randomness and Spike Milligan.  I also find 140 characters too short, and 3,000 words too long.  If there was a free academic journal for 500 word articles I would write for it.

So, my New Year's Resolution was too try and keep this up and keep it interesting.  The fact that it took 18 days to write this is probably not a good sign, but I am editing the thesis at the moment, and I had forgotten just how much I hated editing.

Until next time.
PS I do  myself an injustice, I appear to have more than 12 twitter followers now.  Not that I'm counting.   @richard_hallows

Monday, October 2, 2017

And that's the last of the DPhil Interviews completed...

It feels momentous, but probably isn't, but today, the last of the DPhil interviews were finished.  This does bring some major factors into consideration, the main one being that I have close to twenty hours of recordings to transcribe.

I can feel carpal tunnel setting in already.

On a serious note, some very busy people have given generously of their time, to what was in most cases a complete stranger asking odd questions for which the potential benefit to themselves was probably nothing except the chance to have a chat about things that interest them and help someone out.

It is good that so many were prepared to talk to me.  Of course, I should remember that an equally large number said no politely, failed to turn up for agreed interviews, stopped answering phone calls and emails, or just told me to go away.  I know how those poor chaps providing unsolicited Microsoft support from a shed in India feel...... 

However, the memory I shall choose to retain is that there are some really good people out there, and I have been lucky in that everyone I have interviewed has been extremely likable - as well as clearly knowing their stuff and being prepared to share their knowledge and expertise.  Maybe that is a level of self selection - only the nice people get interviewed?

Despite dreading this particular aspect of the research (generally being far more comfortable in a dark room with a computer)  I have really enjoyed all the interviews.

So thank you to everyone who helped.  It is greatly appreciated.

(Cross posted from DPhil research record.)

Tuesday, September 5, 2017

Cyber Security and the Retail Sector - A Prize Winning Essay

As a general rule I don't tend to do a lot of 'academic' stuff, but for reasons too arcane to go into I entered a competition being run by the British Retail Consortium to write an essay on the cyber challenges faced by the retail industry.   Now, the essay didn't win, but it did come second. which is OK by me because it means I don't have to go and present it as a paper, but I can call it a prize winning essay.  It reminds me that the last time I was up for a prize for writing I was beaten to it by Ruth Rendell (another long story), but thanks to the BRC for running a competition like this.  It was good to spend a week looking at something immediate and relevant rather than purely academic.

Sadly, I don't think I can use this in any way in my thesis, so I still have 100 thousand words to go, minus the 500 I have written this week - so that's 99,500 words to go.  Anyway, I have pasted the essay below for those who enjoy this kind of thing.  Apologies for the typos and any failure of formatting in the copy and paste!

The cyber-security risks facing the UK retail industry are significant in terms of both potential impact and the likelihood of an attack taking place.  This is due to a combination of the attractiveness of the retail sector as a target for cyber-attack and the vulnerabilities that exist to be exploited by any attacker.
The potential impact of these cyber-security risks receives regular media coverage, and includes high profile retail victims such as Target[1] and Home Depot[2] where the costs of those attacks are estimated at more than a billion dollars in the case of Target made up of a combination of litigation, fines, and technical costs.  
It can be argued that some of the business impacts have so far been hidden either by not being part of the headline cost or costs have been externalised for example the cost to financial institutions of card re-issue (estimated at more than $200 million in the case of the Target event[3].) Deloittes estimated that up to 90% of the cost of a cyber-attack remained hidden.[4] IT may also be that the impact of cyber-security has not yet achieved an appropriate level of attention due to the much higher direct financial impact of customer theft in the sector compared to the 5% associated with cyber-crime.[5]
The likelihood of a damaging cyber-security event taking place is governed by three key factors. The potential value of the retail industry as a target for threat actors, the vulnerability of the retail industry, and the capability of the threat actors to exploit the vulnerabilities.
The retail industry represents a hugely valuable target to a range of threat actors in cyber-space, including cyber-criminals interested in card credentials and identity data to enable fraud and theft; ‘hacktivists’ for whom the highly public nature of the retail sector makes it an ideal target for politically or ethically motivated action that might include cyber-attack or the use of cyberspace to otherwise damage a business[6]; cyber-terrorists for whom the disruption of the food supply through attacks on the transport or the retail sector may be seen as a means for instilling fear in the population.
Other business impacts of a cyber-security failing may also include punitive fines of up to 4% of turnover under the General Data Protection Regulations (GDPR), litigation costs due to employee and customer harm caused by data loss, and the reputational damage of a successful cyber-attack that is estimated to decrease the value of a company by an average of 1.8%.[7]
Retail presents some unique challenges in terms of managing vulnerabilities.   Firstly, the retail industry has a very high dependency on technology, both in bricks and mortar stores and online, and encompassing the whole of the retail supply chain including warehousing and transport.  The retail sector is heavily interconnected with other sectors and vulnerable to viral malware attacks that might originate from ‘trusted’ sources. Simple attacks such as ransomware could carry a significant business cost both in terms of lost operational capability due to system unavailability and increased costs for removing the malware infection.
The retail sector has a huge dependency on the infrastructure services from telecommunications and Internet Service Providers through to GPS satellites and the power grid, and the industry would need to decide what constitutes an acceptable level of risk associated with these dependencies and maintain awareness of infrastructural issues that may impact the sector.
However, of more immediate concern would be the vulnerabilities within the retail environment itself.  These include the network and system connections with external partners; an Information Technology estate that includes EPOS terminals that may not be running on updated levels of software; an environment where potentially malicious actors can easily obtain physical access to the EPOS terminals (many of which remain equipped with USB ports that can be easily exploited); unsophisticated users of online shopping sites; employees potentially unaware of the risks in store; the ‘insider threat’ from a disgruntled employee; and, the case of small retailers, an environment where there is unlikely to be any easily available IT support capability.
The combination of being a high-value target, the potential business impact, and a highly vulnerable environment adds up to significant cyber-risk.
Addressing these risks will require a partnership involving industry, government and law enforcement.  The remainder of this essay offers and initial seven recommendation for action.
1.      Cyber-security to be agreed to be a mission critical element of the business with board level representation in major retailers.
2.      Adopt an industry wide approach to cyber-security. This is a shared risk where many attacks are opportunistic in that they look for a weakness in any target rather than a specific target. An industry wide risk assessment may be an effective first step.
3.      Get the basics right, including software levels and patching, data back-up, and encryption of key data at rest and in motion.  Strategically, by following a set of guidelines such as those provided by NIST[8], many cyber-risks can be mitigated.   Cyber-security is an ongoing process with no absolute victories.
4.      Ensure the retail sector is explicitly included within the remit of the National Cyber Security Strategy[9] and identified as being of Critical National Importance.[10]
5.      Ensure effective threat intelligence is available.  Organisations such as the Retail Cyber-Intelligence Sharing Centre (R-CISC) may be a reasonable model - adjusted for a UK focus.[11]  This threat intelligence needs to specific to the UK retail industry and shared effectively with law enforcement and across the industry and may be extended to be an ‘Action Fraud for Business’.
6.      Extend and expand sector specific education and information such as the BRCs Cyber Security Toolkit.[12]
7.      Solutions for small retailers to be encouraged from within the cyber security industry.
These recommendations should be read in the context that cyber-security is an ongoing process and there are no absolute solutions to the risk. Attackers will adapt to any defensive measure adopted by the sector, and risk reduction and mitigation remains the main focus at this time.

[6] This has been seen in the cyber-driven Electronic Intafada targeting Sodastream and social media driven campaigns such as the #grabyourwallet targeting the Ivanka brand.
[8] National Institute of Standards & Technology Framework for Improving Critical Infrastructure Cybersecurity. Available at downloaded on 14th May 2017.
[9] National Cyber Security Strategy 2016 – 2021 available at downloaded on 14th May 2017.
[10] The retail industry is unique as a sector in terms of its importance to the daily lives of the population of the UK both as an employer and as a provider of goods and services, but as a sector, is not specified as a part of the critical national infrastructure or explicitly referenced within the 2016 Cyber Security Strategy except as a participant in the ‘Cyber-Aware’ campaign aimed at small businesses.
[11] It should be noted that even in the US the R-CISC appears to be less mature than the Financial Services CISC possibly does not fully reflect best practice.
[12] BRC Cyber Security Toolkit available at accessed on 14th May 2017.

Saturday, May 20, 2017

Curiouser and curiouser

Some reports suggest that it was not spread by a phishing email[1] (although some hedge their bets by saying that it ‘could’ be, with some suggestions that the attackers had a pre-existing foothold that allowed the initial infection to occur,[2] or that it was through infected websites,[3] although there are also reports based on Darktrace information that it was initiated by a phishing email[4] and an initial email infection in Europe was reported as the source by the FT[5], although phishing is then conspicuous by its absence from the Darktrace blog of 17th May.[6] 

It is probably true that there has been more than a little fear, uncertainty and doubt around the attack vector in particular.

This continues as of today (20th May 2017) with the Register quotes Malwarebytes definitively that the vulnerability was exploited by ports canning for exposed SMB ports and not through phishing emails.[7]

The same report suggests that Windows XP does not now seem to have been impacted (it’s so out of date that even the malware won’t work on it) and it was Windows 7 more at risk.

For me, this does of course beg the question as to why Microsoft rushed out a patch for XP?  And then (for me at least) a secondary question as to how they tested the patch if the malware wouldn’t run on the system they just patched?  Maybe I’m just old fashioned.

In the same report comes the statement that the code that could have led to this exploit being loaded onto Github to work with Metasploit.  (For the conspiracy theorists among you, worth noting that Github is also used by GCHQ to upload ‘benign’ open source tools). 

So, obvious questions. 

Was the Github code used to create the wannacry exploit?

Who uploaded the tool on Github?

Why did it take six days for anyone to notice?

"Curiouser and curiouser," said Alice.


[3] Woollaston, Victoria, WannaCry ransomware: what is it and how to protect yourself available at accessed on 19/05/2017 at 10.25


[6] Tsonchev, Andrew, WannaCry: Darktrace’s response to the global ransomware campaign 17/05/2017 available at accessed on 19/05/2017 at 09:50

Friday, May 19, 2017

Thoughts on the wannacry virus and the importance of starting assumptions

This was written just as a piece to see what happens to the potential outcome of a piece of analysis when basic assumptions change.  I’m not admitting whether I think any of the below is accurate (except for the facts I have taken from others’ primary forensic analysis).
More than enough has been written about the wannacry ransom attack, especially from a technical view point, and the post-attack analysis has made interesting reading in terms of ‘how’ this attack took place.  The quality of the forensic analysis has been pretty impressive – although some elements around attack vector still seem contested (or at least confusing to me.)
The reading on ‘who’ fashioned this attack has been less interesting in that it seems to have come to the conclusion it was North Korea on the basis of re-used code blocks and not much else from what I have read. 
The area that seems to have had even less analysis seems to be ‘why’.  It’s ransomware. it’s to make money.  I can almost see some of the people I know rolling their eyes in quiet desperation. Bear with me.  Why can sometimes help with the ‘who’ on the basis of ‘cui bono’ if nothing else.
What has struck me however is the fact that much of the discussion seems to have been based on the unquestioned assumption that this was about money. But something doesn’t seem to quite add up.
In the case of wannacry the problem is that if ransomware is to make money it’s just not very good ransomware – despite using what I understand to be an innovative and highly effective propagation mechanism (although more on that later).
So why isn’t it very good ransomware?  Well, firstly, the estimates I have seen suggest they have not made very much money out of it.  If that is the most effective measure of ransomware success, then wannacry would seem to fail.
Secondly, the inclusion of what seems to be a poorly thought out kill switch, which seems to have been the main reason why the spread was contained.  There are suggestions that this was a badly designed sandbox detection mechanism[1] and not a kill switch at all, but nevertheless, it has allowed the malware spread to stop.  This does raise the question of why a variant appeared that included a different domain name acting in the same way as the first. If it was clear that this technique was preventing success, then sending out a variant with the same technique embedded within it makes little sense. What I haven’t seen is whether there is any indication that any new variant is from the same criminals or whether it is someone taking a chance on getting a few dollars for not much work.
Third, the payment mechanism wasn’t particularly good with a limited number of hard coded bitcoin wallets (some reports say four and some say three) that would mean that any correlation between someone who paid and decryption would need to be done manually.   There seem to be two main public explanations for this; technical ineptitude or no intention ever to provide the promised decryption. The whole business model of kidnap and ransom (physical or virtual) would seem to be based on the belief that payment of the ransom will be honoured by the criminal.  Certainly in a repeatable virtual crime this would be fundamental to determining financial success.  This ransomware attack does not appear to have been designed to achieve financial success – at least not through ransom.
Of course, if we assume a financial motivation (and let’s at least be aware that it is an assumption) there are other ways to make money out of this sort of event.  Cyber-security company stocks, predictably, shot up in the immediate aftermath[2] (but have since fallen back quite a lot) and a plan that involved profiting from this would make some sense. I can only assume that somebody in law enforcement is already looking for unusual trades prior to the attack.  (I am not going to go down the route of suggesting that it was the cyber-security firms themselves who were responsible – most of them already seem to have a licence to print money – but any proper analysis should consider this possibility.)
However, we really should consider the hypothesis that this attack was not about making money in the first place.
The assumption that it is financial has led to the conclusion that it is a technically inept group responsible. 
However, if we assume a technically capable group as the perpetrators (rather than assume financial gain from ransom as the motive) then things potentially look somewhat different. With this new assumption:
·         Could the kill-switch be a deliberate design point to limit the spread of the malware - only disguised as a poorly thought out detection evasion system?
·         Instead of being a technical disaster, is the non-functional payment mechanism a deliberate attempt to damage the ransomware brand?
·         And the sloppy coding technique, is that just deliberate obfuscation of technical skill?
·         Could the choice of an exploit that took advantage of SMB and port 445 suggest that home users were explicitly excluded as a target?
The (welcome) rush to patch was of course also predictable and Microsoft had released a patch a month earlier (so a lot of scheduled upgrades would have taken place by the time the attack hit) and the attack did conveniently avoid year end, quarter end, or month end periods where the capability to implement changes might have been process limited. Microsoft’s ability to produce an XP patch in short order was clearly helpful but surprising given it has been out of support for so long.
It could also be argued that the one thing that wannacry has achieved above all else is raising awareness of the dangers of out of date and unpatched systems and as a result this could easily by the best thing to happen to UK IT infrastructure since we gave up on state support for ICL (if you’re not as old as me then you will have to look them up). Certainly the government’s NCSC seem to think it had value as a communications programme to “make people sit up and take notice...”[3] This must have been a relief, after the time and effort they have spent trying to get the message across both regarding the need for basis patching and the likelihood of a major event.
Once we change the assumption of financial motivation to an assumption of a high level of technical capability then there are completely new possibilities as to motivation and perpetrators.

Wednesday, March 29, 2017

Instruments of Darkness

Recently, completely by accident, I picked up a copy of Alfred Price's Instruments of Darkness: The History of Electronic Warfare which (in the main) tells the story of the competitive development of radar capability during World War Two.   (It does extend into the 1960's but you can tell his heart really wasn't in the telling of the events in the later years.)

Regardless, it is an absolutely fantastic book that is as gripping as any novel - despite the subject matter being basically the history of the development of techniques for electronic detection and deception.  One of the elements that was interesting for me was just how relevant some of this story was to the world of cyber-security in the twenty-first century.  (I admit this could be just my way of trying to pretend that the time spent reading it really was effective research, but I don't think so.)

Anyway, some of the key messages:

  1. There are no absolute victories. At best, the developments provided a temporary advantage in a see-saw battle of measures and counter-measures.   The development of new solutions had to be constant.
  2. A combination of offensive and defensive measures were required.
  3. Sometimes your own capabilities could be used against you - for example signals emitted by electronic counter-measures on some fighters were used to track those fighters.
  4. Not all electronic measures were countered electronically - sometimes a change of physical tactics was the most effective response (night fighters infiltrating the bomber stream for example).  
  5. Electronic warfare was very much hybrid warfare with kinetic attacks on radar stations and the like.
  6. Technical skills combined with management that got the job done was a major source of differentiation.
  7. Intelligence regarding the enemy's capability was fundamental to success, including information from PoWs, acquisition of enemy equipment (hint: never put secret kit in a location with easy access for a commando raid), creative interpretation of seemingly meaningless breadcrumbs of data that helped provide a strategic picture.
  8. There were mistakes, dead ends, ideas that should  have worked but didn't - and ideas that should not have worked but did!  It was not a predictable environment and luck plays a part.
  9. Getting it wrong was very costly.

I guess I shouldn't be surprised at some of the similarities as cyber is really the mutant child of electronic warfare and information warfare so it should have some of the same characteristics, but I did find it bizarre that there were even competitions to generate new ideas and to get new developments moving (not unlike some of what is being done today for cyber).

I must admit though that in general I am not keen on extrapolating from history for an understanding of cyber issues, but in this case I will make an exception.
(Cross posted from my DPhil blog as it seems relevant.)