The Nightmare of IOT Vulnerabilities - An Interview with Stefano Zanero & Roberto Clapis

IOT Device Security with With Stefano Zanero and Roberto Clapis.

On today’s episode we host a conversation with Roberto Clapis and Stefano Zanero from Secure Network in Milan. We tackle the issue of IOT device security and try to break down just where companies and users are at with this issue currently. We get a background to Stefano and Roberto’s work and their interest in security as well as little peak inside their presentation from Art Into Science. One of the main takeaways from the discussion is the idea of communication between security and other sectors, something that our guests suggest would greatly improve the strength of security. Listen in to hear what they have to say!

Key Points From This Episode:

• How Stefano and Roberto got into security and came to their current positions.
• Their relationship with the Black Hat Conference and their presentation.
• How our guests approach teaching about vulnerabilities on IOT devices.
• The introduction of IOT devices and the increase of vulnerability that comes with this.
• Changes in the field, the importance of updates and isolated networks.
• Examples of good cases of security according to Stefano and Roberto.
• The desperate need for more communication between security experts and developers.
• How GDPR has been affecting Europe, and the reason for its recent rise in popularity.
• And much more!

Links Mentioned in Today’s Episode:
Secure Network — https://www.securenetwork.it/it/
Politecnico Milano — https://www.polimi.it/en/home/
Black Hat — https://www.blackhat.com
Codemotion — https://codemotionworld.com/
Mudge Zatko — https://hackstory.net/Mudge

Introduction:
Welcome to another edition of Cyber Security Dispatch where we talk with experts and practitioners who are pushing the envelope in cyber security. This is your host Andy Anderson. In this episode, the nightmare of IOT vulnerabilities, we talk with Stefano Zanero and Roberto Clapis of Secure Network. Stefano and Roberto walk us through the nightmare that is security, more telling perhaps the lack of security in the IOT devices we’re making these days.

Transcript:

Andy Anderson: Let’s just start, just introduce yourself, you say your name, where you’re from. Just so we have it on tape and stuff.

Stefano Zanero: Sure, I’m Stefano Zanero, I’m an Associate Professor at Politecnico Milano and I founded the Secure Network which is Italy’s largest penetration testing team.

RC: I’m Roberto Clapis and I work in Secure Network which well, you already know.

AA: Yeah, you guys worked together. Awesome, well thanks for having us, being on the show with us, always fun to chat with people particularly who are kind of outside the US and maybe seeing some of the issues that here we may not be encountering. I always think it’s interesting just to hear kind of how you got into the space because so far I don’t know any kid who woke up at age five and was like, I want to work in security.

SZ: As far as I’m concerned, I woke up at age five and I decided I liked computers. But setting that aside, I think that what got me interested in security originally was the interest in network protocols. Experimenting with network protocols I found out the world is interesting things like you could make computers disconnect from the internet at some point.

I got interested and hooked into that and I started studying in the university which was not teaching any computer security course, in fact, I started teaching it after being hired. It worked the other way around, I went to the university with the course from the university. That’s my story.

AA: Right. There’s some quote about wanting to foresee the future, you see doing better. You’ve been inventing your career path at least. How about you?

RC: I wanted to invent stuff and make stuff and the easiest way to make stuff was to program because I started as a coder but after a while, coding and understanding the internals of how things worked, I started to look to repeat them. I noticed stuff that shouldn’t work in some ways and started to see weird behaviors.

You know, that’s funny effect when something behaves almost as it should but that almost allows you to do something more with it and I got curious and I liked it so – he actually asked me to work for him so I got the chance.

SZ: It is my fault.

AA: I’m sure he reminds you that.

RC: Yes, every single time, yeah.

AA: Cool, you know, you guys flew from a ways right? You might win the price for like farthest travel to this conference.

SZ: Possible? There’s also people from Israel, we’re kind of tied.

AA: Might have. Other than coming for the food in Austin, what brought you guys to this conference?

RC: I was actually among the people that has started putting this together last year and I think that having a conference that is focused on defensive techniques is actually very good thing. I’ve been on the Black Hat review board for years and we have actually added also tracks in black hat dedicated to the fans because that needs to be done and a lot of the conferences are really focused on the offense, which is good.

Offense is like the mother of the security industry in many ways. But defenses, I mean, there’s way more people that who’s day job is in defense so the offensive side is in part called, in part necessary, the defensive side is very good.

AA: Yeah. You guys specifically – you know, you haven’t spoken yet and will definitely post kind of anything that you have, the recordings et cetera, so people can see that but for people who may not see that, what’s the sort of things that you’re talking about today?

SZ: Today we will talk about IOT. We started looking into the code for IOT device two years back and we started to see vulnerabilities that shouldn’t be there and by shouldn’t be there, I’m not talking about any vulnerabilities, I’m talking about buffer flows and for much strings that any modern text compiler complains about.

AA: Yeah.

SZ: We started wondering why is IOT 20 years back?

RC: Are actually older than him.

SZ: Yeah, most of the vulnerabilities are older than me.

AA: Vulnerabilities.

SZ: We started looking into the reasons for that and we found that there is no tool chain, no way for us to teach people that are programming IOT devices to write secure code, there’s no communication there.

RC: Basically what one of the things we did during this research was to reach out the security community and say, “Hey, okay, you can give and advice, an actionable advice to somebody developing for the systems, a developer, not the security expert and developer.”

ou want to make it like  a most valuable advice, tweet long advice, what is it. And got the most offer responses. Not awfully the sense that they were bad, they were extremely good. But most of the responses were like, “Okay, you need to sit down and start track manually. Then when you have more to your frats, you can begin building in your defences and apply your language based,” which is all things that I would teach but if you read them with the point of view of somebody needs to program the next intelligent term of stat or the next baby monitor, it just not going to work, it’s not the way people are – it’s completely impractical for them to do that.

We don’t have good suggestions such as, okay, just to start, you need to knock into this, filter this things and use these libraries and that’s going to take care of like 85, 90% of the problem, we don’t have that. It’s not going to work for 100% of the cases but we don’t have yet 90%.

AA: Yeah, I mean, we were talking about this a little last night, it’s sort of like, the two worlds are just completely siloed, right? They’re not touching each other at all and so when they try and communicate, it’s just – it’s like ships passing in the night. We’re talking about this tuff and you're talking about this stuff and it’s just not really resonating with each other.

I think that’s really interesting, this part of this talk essentially like the top five - if you’re going to do nothing else, do these five things, right?

RC: We also have apart from that, mostly because these talk that we’re going to give tomorrow is going to go around in developer conferences. We want to communicate, for most people that are looking for the defense strategies which like this conference and also for conferences like Codemotion or auto that our four developers, they’re meant for developers to – yeah, we have a checklist.

AA: Yeah. You’re just going to start showing up.

RC: Exactly. Basically, the theme of the talk is actually, we are lying, it’s not about vulnerabilities, it’s about zombies, the old talk is about zombies and how to kill those zombies. So we have a list of like the bad tools like you see in zombie films, the guy trying to shoot at zombies, that doesn’t work because you need to bat them down. As everybody knows by now. You know, the guys in the film never know. The developers do not know.

We look at this thing, the security profession and say yeah, no, how could you possibly write that? The reason is that the guys on the other hand, on the other side of the keyboard do not know. We don’t give them the ways to do that, actually, the two of us have tried to configure one of the development environments that these guys are using to, in order to put in the security flags that we are used to have in every other development environment. It took us hours to figure out how to reconfigure it so that it was just as secure as GCC with the default options.

AA: Yeah.

SZ: We’re not talking rocket science security stuff.

AA: It’s the basics.

SZ: We actually didn’t manage, for some of this.

RC: There’s just no equivalent, there’s some options that you are just left wondering and you don’t know what to –

AA: It’s problems you can’t, if you didn’t build it, you don’t know, it’s sort of like –

SZ: No, you would need to patch the environment to enable stack protection for example. There is no other way.

AA: You know, having not, for those of us who are daily kind of thinking about either developing or programming for IOT devices or thinking about the security posture, is it – are they patchable? Let’s say I bought a camera that has a bunch of these vulnerabilities in there, is there a refresh on the systems?

RC: Most of the IOT devices that also the program that you cannot really deliver patches very well. That would be a good reason to do it right the first time.

AA: Because there’s no interface to it, there’s no like –

RC: Yeah, one of the things that we noticed is that besides the actual vulnerabilities, there’s a number of things in operating systems nowadays, stack protection, ASLR, that do difference in that and make a little bit harder the work of people exploiting stuff.

In most of these devices, those defenses are just not there. And that’s way more difficult to enable and I’m not just talking about your regular smart television at home. I’m talking about devices such as a half-ton investor robot that is working with buffer overflows that are straight from the 90s.

You could pick up all the ones, matching these type for front end profit and literally code pay line by line what he was doing in 1994 and it would work perfectly on today’s robots.

AA: That truly sort of nightmare scenario that you know, every none technical magazine and newspaper talks about like a machine’s attacking us, robots sort of –

SZ: As long as the robots are like that, I think there’s no problem, we can shove them down, right?

RC: Yeah, you can just exploit them to death. If you see any terminators walking around in here

SZ: We form a really longer quest to it, probably going to go down.

AA: Ask a long question and just watch its –

RC: A very long question and see.

AA: That’s incredible. When you sort of think about – when people are introducing these IOT devices into their networks, right? You had a traditional network or maybe you had some mainframes and you know, desktops but now suddenly, we’ve all seen the graphs where there’s sort of the number devices that are networked is just exploding, right? I’ve seen 50 billion devices by maybe 2020, right? Which is like you know.

SZ: Disquieting.

AA: Yes, it’s troubling, right? You know, when you think about that, like what does that mean from a security perspective for that whole network, right? Because you’ve suddenly introduced these devices that are – you know, have huge vulnerabilities, are they essentially just easy gateways into that network, easy ways to kind of like do lateral attacks and whatnot?

RC: The short answer is yes.

SZ: Well, the other answer is queue to rob, running away from this room just screaming.

RC: Yeah, I think for most of the devices we are talking about, the only way to secure them would be to just keep them offline because there are no patches, the vendors are not interested in patching them. There is no way to create a security standard. For example, for energy consumption, you have a label telling you these device from this amount of energy but that’s not true for security.

Sell a device because it’s secure, no one is going to care. So vendors don’t do that, don’t usually do that.

SZ: That’s the what Mudge was proposing to do an underwriter’s lab for security which is actually kind of a good idea right there. At least the basics.

AA: Who is it?

SZ: Mudge Zatko, it was proposing to do like – let’s do the same thing as the underwriter’s lab for electrical stuff, these are the same for cyber security, it’s not going to say they are secure but you are going to see at least we tested them for these basic things and that they don’t really break down that easily.

That would probably be in the direction of what that was saying but then I think that the actual answer is we want to make sure that whatever devices get deployed can be updated. Then there’s the whole problem of how we create the business model around this things so that the companies that produce them do not just go out of business and leave millions or billions of these devices rotting around.

AA: The zombies, right?

RC: The zombies, yes, kept coming out of the graves. That’s a different thing but first of all, we need to figure out a way to make them updatable which is not always easy. I mean, if you think of industrial stuff, if you think of healthcare related stuff. There’s things getting planted in people’s bodies.

SZ: Plus, we want to create libraries and standards to update stuff because the update mechanism, if that is broken, that is going to break everything down and we have seen really poorly implemented updates.

AA: You know, we can always sort of try and predict what’s coming down the pipe but you know, the history show that we’re not particularly good at it, right?

SZ: It’s difficult to make predictions.

AA: I think that idea of leaving, making sure that you’ve left a pathway to be able to come back and address problems that might not be there.

Are you seeing any one sort of understanding the implications of those IOT devices, essentially maybe networking them but putting them on a separate network or you know, sort of trying to isolate their access? Is that something that’s – you’re seeing people do or is that sort of like –

RC: It depends on the field. In the industrial world, that’s been for a long time on the answer so the reason why these things were not vulnerable or they were vulnerable but not really attacked was that they were disconnected from the internet and even now, I mean, I was giving a talk at the conference about industrial systems and one of the guys said, “Okay, so what do I do? I mean fine, the problem is clear how do I deal with it?” It’s like separation of methods as we do already going to answer that and the answer is yes partially. Of course it helps.

AA: Nothing is a fantasy.

SZ: Yeah because the reason is that most of these devices are more and more dependent on connecting to the outside. So you need to pull course in your perimeter again, just like it happened on the corporate metrics right. We have all of the DYOGN, all of the connectivity to services, pull course and singles for physical devices.

AA: Yeah so you need to be able to phone home to manufacture, to the service provider whoever that is, right? So you start that idea of air gapping, sort of?

RC: Actually there are several security cameras or other IOT devices being the same network with the machinery which was insulated but the point is that once an attacker gets in, you can hack a cheap camera without very basic exploit. Then from there, try to escalate or do any last minute movement because maybe an industrial robot is not something you can easily find but a cheap camera you can probably find on the market.

So you can exploit that first and just stay there. Since those devices need internet access you’re pretty sure you are going to be able to reverse all of that once you get in.

AA: You know we try not to make these interviews all sort of the sky is falling, right? Full things right? I know in this space it’s hard not to. Is anyone you know kind of doing things well or is there anyone you would point to be as much as we point to the problems but kind of the examples?

SZ: Yeah, actually since we started working on will, some of our clients on these problems they started noticing that it was an issue and they had to address it. So we started to both test some of their tools and some of the other part of their code and once we found a pattern and patterns that they were getting wrong, we started giving courses, giving classes to their developers and their response and the feedback is pretty good. Because the company realized that it’s way easier to fix something before it goes to production and they want to do things right, so yes, things are moving. I hope they’re going to catch up. There’s something.

RC: So one of the things that I find in the light of trying to figure out the silver lining in effect.

AA: Right, there is one I promise.

RC: There is one I promise. So no, one of the things that we actually found very good was that while we were doing research in industrial robots for instance, we got in touch with one of the largest robotic vendors. I mean if people google it up it’s not difficult to figure out which one and what surprised me positively as the time of reaction. They had a single email for sending in full reports for all their staff.

We have seen a very large company with very complex teams is actually very good because you don’t need to figure out who to record it and they’ve replied within 24 hours and their right team engaged within 48 hours. Within five days, they came back with a remediation plan and the only thing that they were worried about, well didn’t see what it was which is was kind of good and the only thing that they were worried about rightly so was the timeframe because they said, “Okay we are deploying this on robots, we cannot do it in 15 days. We need to have time.”

Once we told them that we had months before the conference, we wanted to present the report at, they were absolutely fine with that. They deployed the patch and made and cared about the fact that their own implementers, vendors, consultants in their partner to knew that it was important. So the are handling carefully everything at least for some of these companies it is actually pretty good. They’ve learned from the market and they started doing it the right way.

AA: Yeah, the fear of the front page of the newspaper is I think is a significant one for a lot of these right?

SZ: Yeah, the weird thing is that they wanted us to say their name, to put their name into our slides. So when we present our talk tomorrow, they’re going to be there and they admitted that, “Yes, we are doing everything and we did what we could to fix it.” Which is a really nice response for sure.

AA: Yeah and you know, in a free market you hope eventually sort of begins to sort out who is doing the right things and maybe not. There should be challenges because they go out of business and then there’s no way to patch these things, right?

Zombies, cool. You know what else, what other things, the stage is yours what else do you want to talk about? What message would you get out to the community?

SZ: So the security community needs to start talking with developers because what I have seen, one of the reasons I’m here is that this conference is about defense because in security conference, you mostly see security people wanting to see the new cool attack and then obviously, it must be done because otherwise we are going to stay in the past.

We need to do that but there is one more thing that we need to do is that what may seem obvious for us, it’s not for most of the developer community, so we should start communicating with them. We should start telling them how to do stuff and why, mostly why. The how will come as a question after that.

AA: Yeah. It does seem like anywhere, you like to chase the shiny new toy, right? Like, “Oh this is cool let’s talk about that” but then some of the basics that are not as exciting but that’s –

RC: Yeah and also I think one of the things that I’ve learned from doing this research was that we really have a problem with enable. So we need to talk to the developers but actually what we do need is to put into their tool chain things that enable them to understand and to fix stuff and some of these things are not rocket science. They could be there, obviously they are not there because they are not requested.

So if you are developing an IDE, your customer with the developer wants to be faster, they want it to be easier to share their knowledge. Security is not probably going to be their concern. We need to find a way to inject in this market. The security considerations in there because once the people that create IDs, SDKs that created the base on which the IOT is built. Put in some of the building blocks, those building blocks are going to go a great length into making IOT security at least as bad as a security in other sides of the industry which would already be an improvement.

AA: It will just be, you know, adds to the rest of us.

RC: Which is better than the rest of us.

SZ: Yeah, I mean while we were thinking that you couldn’t and we were concerned about the spectrum meltdown, we were also trying to tell one of the people we interviewed from IT, why is this thing could be an issue. So that’s the reason of the gap there.

AA: Right.

RC: There is a gap or people writing and they are saying, “Okay so do we need to care about this all now that is your expectation in your IT?” “Not really, but yes but these other set of things that probably come first. “

AA: Yeah that’s like about a dozen.

RC: Yes.

AA: Very cool, partly we, you know I am always interested because it is the area that we, for the other business that we are in think about is, you know, the idea of moving target to fence is that something that you have encountered or seen, particularly you guys come from sort of an academic background, what’s your exposure into that?

SZ: So I think it’s an idea that who’s time probably has come some ways. There’s all these ideas moving target defense, deception, things that have been in the community. They have been spoken about for years but not really practiced that much. In the IOT world, I don’t think they are of the maturity level to consider probably not yet but it’s surely an interesting subject and it is a subject that I am hearing talking about more and more.

AA: Cool, the other piece that I know that you’ve spoken about before and just because you’re background being in Europe, you know GDPR is sort of like every conference we go to, someone is sort of screaming from the top of their lungs that we need to comply with this. What’s your perspective on that, the history of it? And I think I heard you talking about how it’s maybe not as new as some people would imagine it is.

SZ: Yes, so the reason why I was smiling while you were saying comply and every time I hear somebody talking about compliance, my mind goes to the Borgs for Star Trek, comply. So imaging the people from Brussels as to Borg for Star Trek, it’s kind of fascinating. But so, I think that you have a very right observation right there that this is not something new. Most of these regulations have been and effecting Europe before like 20 years almost.

So the reason why this is new is that the way that this has been applied now impacts strongly also companies that are not based in EU but the tar at treating data related to use of business and which in a globalized world it’s basically every single company that has business on the internet. So I can understand the screaming, I can understand that.

AA: Four percent of turnover gets a lot of people kind of excited for this.

RC: Right, yes of course. On the other hand, I wouldn’t be that scared about the sanctions or the fines themselves because it is a very long process together to be fine about that but what I think is important is that of course, GDPR changes perspective for American companies because the way that the protection of personal data is structured and they use it very different, radically different.

So it is actually more of a problem of legal and procedural issues than of security issues per se. There’s not that much technical things that are different or required. There’s requirements for instance in separation of data but it lies more in the programmer’s ballpark and in the system designer ballpark than in the security area as we usually define it.

AA: So where the data is residing and how it’s transferred not necessarily like the –

RC: Right, how it is handled. I would just separate it, all of the request about being able to explain algorithms used for selections or things like that but they are all requests that relate with IT, of course, not necessarily with security. And in fact I think that this security, pure – I mean of course, for security are thinking of the seesaw, of course they are going to be heavily involved but the technical teams doing security, I don’t really know if they are going to be involved that much.

AA: Cool.

SZ: Plus if you actually read what GDPR is asking companies to comply with, that sounds, like, reasonable. I mean it’s not something – well if you read through it, when I was reading it I was like, “Well yeah, I hope so. I hope these apply to my personal latest.” So yeah, that is not impossible to share but it is something that should be done so.

AA: Very cool and do you sort of see – I mean I think what is interesting is will that effectively become a standard, a worldwide standard, right? Because if you – for these global companies you’ve got to comply with what’s happening in Europe like it’s crazy to be thinking about sort of operating in a different way, in a different region right?

RC: It makes a lot of sense and as I was observing through a meeting of CISO's a few months back when we were discussing this, I was like, “Yeah I see you now, we got hyped up from who gets somebody’s socks late. It is going the other way around.

AA: It’s payback time.

RC: It’s just payback time.

AA: I agree with that, I can’t think of a better ending right?

RC: Yeah.

AA: Finally Europe is coming back around, right.