INSIDE SECRETS FROM A NETWORK PENTESTER
About this episode
In this episode, special guest and network penetration tester Mike Sheward joins us to talk about ethical hacking and penetration testing. When many people think of their dream cybersecurity roles, they initially think of penetration testing, so this is a great episode to learn what that career is really like from a practitioner in the field.
First, Mike will share how he ended up as a penetration tester and determined that was the right role for him. He also will tell us the differences between working for a small and large firm when working as a penetration tester. Each has its own advantages and disadvantages, so it is important to understand them as you begin your job search.
Mike also shares how penetration testing and forensics often work together in the field. Throughout this episode, you will learn how to prepare yourself for a possible career change you might make, what hiring managers look for in an employee, and how having experience and connections can help in your application process for a position.
What you’ll learn
- What penetration testing is
- How to choose the right cloud service for you
- How penetration testing and forensics work together
- How experience and connections can help you in the application process
Relevant websites for this episode
- Your Cyber Path (https://www.yourcyberpath.com)
Other Relevant Episodes
- Episode 15 – Hunt for Jobs Like Pentester
- Episode 21 – Your Reputation Matters to Us
- Episode 40 – Security Awareness and Training
Hi, this is your cyber path. We’re the podcast that helps you get your dream cybersecurity job. I’m Kip Boyle and I’m an experienced hiring manager of cybersecurity professionals. Wes and I are taking some time off, so today we have a special episode for you. Our friend, Mike Sheward is an amazing network penetration tester. He finds vulnerabilities in web apps and in the infrastructure that they’re hosted on. He’s my go-to guy whenever my consulting customers need a pentest.
So recently Mike was super generous and he spent some time with our cyber pathfinders during a weekly office hours segment. And he told them how he got into cybersecurity and he answered lots of their questions. So today we’re going to share this interview with you. Now, before we hear from Mike, I want to remind you about our free guide. It’s called Play to Win, Getting Your Dream Cybersecurity Job, and it describes how taking a capture the flag approach is going to help you compete and win in your job hunting. It’s a really helpful 20 page visual guide, and you can check it out. You can go get a copy for yourself. Just go to yourcyberpath.com/PDF. Yourcyberpath.com/PDF. Check it out. Tell me if you like it. Tell me if you don’t like it because I’ll change it if you’ve got some really great feedback for me. Okay. I really think you’re going to enjoy this episode with Mike. So let’s hear what he’s got to say,
But yeah, essentially I’ve been in information security for about 15 years at this point now. I was born and raised in the UK. I’ve been in the states for about 11 years. So my background is that one of my first jobs was doing network engineering for the local government, local educational authority, where I grew up in Winchester, home to the famous sauce. And I happen to be doing network monitoring one day and I saw on a traffic graphs a bunch of very strange outbound traffic leaving a network that we were in charge of. And so it looked to me… I dug into it a bit. Did a pcap of it. And it looked to me like malware traffic. And so I decided, it was the end of the day and these were all schools that were using the network.
So I decided to just cut the network connection to make sure that there wasn’t anything malicious going on and people were losing data. And so the way I did it was basically shut down the machine that connected that school to the internet remotely. And the next day I planned to go in early and just reach out to the school and make sure everything was okay, which is what I did. And I spoke to the IT person at that school. And he said, “Oh, I think I know what that is. Don’t worry, I’ll go check it out.” He reconnected to the internet and that was it. I didn’t think much more of it. And then about six months later, a person from the educational [inaudible] HR department walks in and says, “Hey, can I talk to you about something?”
And of course I’m like, “Oh, what did I do? What did I say?” And they said, “Hey, so what actually happened was you actually caught that guy hosting things that were very bad in a school environment and the traffic control appeared to be a file sharing traffic. And I was like, “Well, crap, that’s bad, but I’m glad I caught him I guess.” And so, yeah, we had to put all the evidence together and I was like, “Well, that’s actually a really good feeling to know that you noticed something and you stopped something very bad from happening. And now a bad person is in trouble.” And that’s what set me off on the journey from being a network engineer to where I am now and everything along the way.
So what happened from there was I decided to go and do more security, specific things. This is about the 2007, 2008 timeframe. And so I ended up coming across to Dallas. This if from the UK. It was still fairly expensive to do training courses over there. So it was cheaper for me to fly to Dallas, stay in a hotel and do a bootcamp training course for CEH and CHFI certified ethical hacker and certified computer hacking forensic investigator. So I came over to Dallas and did that. That’s where I met [Jaf], my future wife and explains how I ended up in the states. I first met her there and then I went back to the UK. I went from local government to central government. I worked in British intelligence for a bit doing defensive stuff. And then I wanted to get more into pentesting and offensive stuff.
So in government work, obviously you’re very boxed in at civil service work. So you’re given a job and a grade and a role and you stick with it and then you have to work your way through the system. So it would have taken a really long time for me to be in the civil service and get to the job I wanted. So I quit and became a private contractor and sat in the same building next to the same people. I could do the job I wanted. And I also got paid for it as well, which was great. So that’s what I did. I worked for a company called Encryption, spelled incorrectly because of domain name reasons. And I ran the pentest. So I was a pentester primarily, but I also ran the digital forensics service that we establish there. And then I was in Encryption for a few years until I finally moved over to get married.
I stayed working for them remotely when I first got to the states. But ultimately when I got to the states, I ended up working for a company called Concur. They do travel and expense management software as a service company. I was the first technical security hire at Concur. So basically I went into an environment where they had a bunch of reasonably good security things all over the place, but no actual formal technical security program. With my background being in a bit of defensive and a bit of offensive security, it was my job to piece that together.
So when I left Concur about four years later after they got acquired by SAP, I was running the global security operations team. It was quite funny because I got asked what’s your… Because I was fairly new and the only person to have the job I was allowed to pick my own job title. So I picked manager of global security operations because I thought it sounded awesome. And it’s quite funny now, five years later seeing jobs come up in that team on LinkedIn at Concur. And it’s like, “I just pulled that name out of nowhere and it’s still there.
So that’s about four years, yeah. Like I say, SAP came in and bought Concur. One of the really nice things about Concur was it was still a fairly small company while I was there. So I still knew most of the people even know it was a global company. But when SAP came in, obviously it becomes a giant company, like 80,000 people company. And typically, the bigger the company and the bigger the security team, the less you get to do because you get siloed into doing a very specific thing.
So that’s like my first pro tip is, if you look at smaller companies and startups for security roles, you’ll probably find yourself doing more different things, more of a generalist than a specialist, just because resources are obviously more constrained. And as I’ve gone through my career, I found that I prefer being a bit more of a generalist and exploring different things. I get bored if I get siloed into one thing. So when SAP obviously came in and they were very specific job titles and roles and things like that, I decided that it wasn’t really for me.
In 2015, I did a couple of different jobs and I wasn’t really a great fan of either of them, to be honest. At Expedia and a startup called [Extra Help]. Extra Help is fine, but I actually ended up getting a tap on the shoulder from [Raj String] who was one of the co-founders of Concur. He was setting up a new company in Seattle called Accolade Healthcare, a tech startup. And he asked me to go do pretty much the same job that I’d done at Concur there. So I went to Accolade and I was there for about another four years and the company IPOed and that exposed me to a lot of medical HIPAA information and things like that.
So it was a good journey and gained a lot of experience. I wanted to take the final step because I thought, I really don’t need sleep anymore. Screw it. So I wanted to become a chief information security officer and have complete control over the program. So I went to a company called Amperity, which is in the same fold, if you like, in Seattle of Madrona backed companies, the Accolade and Concur was, and I was there for a little bit. It wasn’t really a great environment for me, but it was fine because I got offered another position. A company called Particle, which is where I am now. It’s an internet of things startup and I’ve been there since January and I’ve been [inaudible] for a long time, and I’ve now been given some extra responsibilities outside of security, but primarily I’m doing security compliance work and it’s awesome.
Along the way, so I’ve written… I wrote a couple of books. The reason for doing that was not something that I really set out to do. So I wrote a book called the Digital Forensic Diaries, which is basically a collection of short stories based on some of the real world investigations and things that I’ve done over the years, either at Encryption or at Concur. And I basically took actual things that happened and fictionalize them because obviously you can’t go and write things that are covered by NDA.
And I did that simply because I got really pissed off at an episode of Scorpion on CBS, which is… It’s just like a CSI cyber type deal. One of these fancy shows where they show off all these cyber crime things. And the scene that did it for me was there was a scene where the hackers were trying to… Well, the saviors are trying to free an aircraft of a virus by passing an ethernet cable down in the landing gear of a flying jet. And there’s so many things wrong with that. Not least that you can’t get from inside the plane to on the landing gear, because it’s a pressurized structure, but anyway.
So that really annoyed me to the point where I went off and wrote a book, which is what any sensible person would do. So Digital Forensic Diaries is essentially that. It’s basically five short stories about real world things that happened. And they’ve all been fictionalized, like I say.
What happened then was a lady called [Beck Reed] read that book and she worked for a publisher in the UK. Said she grew up not too far from where I grew up in the UK. And they did technical books. And so she reached out to me and said, “Hey, really liked Digital Forensic Diaries,” which I only did as a hobby type thing, just to see how it would turn out. “Would you like to write a technical book, a non-fiction book?” And I said, “Sure.” I’m a bit nervous about doing it because I’ve heard horrible things about doing that. And I typically have a tendency to stop things and then let them drift away and then maybe go back to them. But when you write a book for somebody else, they give you very strict deadlines and all that. But I did it and I gave it a go. I really liked the process and that first book came out in 2018 and it was focused on forensics and incident response.
And then a couple of years pass and earlier this year, we did the same thing again, but with a different book on security operations. And in-between those things I’ve really picked up the… So there’s the non-fiction technical work books that I’ve written that are like brain dumps of things that I’ve done over the years that I don’t get to enjoy so much because I’m always on deadlines and people get angry at me if I miss deadlines. And then there’s the fun exciting writing that I do, which is a book called the Digital Forensic Dairies and then I did some pentests diaries as well. So there’s about three of those out there right now. I have the other two stories in my brain, but very similar to the Digital Forensic Diaries. They’re real-world stories about actual pentests that I’ve done over the years.
Pentesting is so much fun. I really enjoy it. And I did it full-time fairly early on in my career when I worked in the government and encryption. I was probably a better technologists then, because I’d spent a lot more time playing with code and exploring things and tools and things like that. So I think that on paper, I would be a better pentester because I had more time to experiment and play around. But I would say that when I do pentesting now which I still do a little bit of, I have my own company that I do a little pentest projects on the side when and where I can to stay current with it.
So, I feel like I’m a better pentester now because of all the experience that I’ve gained in management roles and I know the weird things that people do that aren’t strictly code or technical things, but instead of… A classic example, was like 10 years ago, I was on site in a government location. And they’d given everybody in the active directory domain admin rights by accident because they added domain users to the main app. And nobody knew about that and it had been like that for years. That was one time when another agency was outsourcing its IT support to another company and in order to give them the ability to manage their active directory, they created an active directory management portal and exposed it to the internet. They just forgot to put authentication on it. So anybody could administer their active directory. That actually made it into one of the pentests diaries.
Things like that you pick up and figure out along the way. But yeah, that’s how my journey played out and how I ended up where I am. I’ve been very fortunate that… I’d say that I’ve taken a couple of wrong turns, Expedia and [inaudible], but I’ve always been able to bounce back fairly quickly in the event of doing that. So, my takeaway is, everything happens for a reason and you have to try different things to make sure that you know you’re in the right place and I’m super stoked with where I’m at now at Particle. I think it’s like the perfect role for me because it’s a young company with solid user base and a lot of room to build a program. And it gives me a chance to be hands-on and work with engineers, but it also gives me a chance to sit in front of customers and do some of the business ey sides of security as well. So it’s a great thing for me at the moment.
Excellent. Thanks Mike for getting it all kicked off. What do you think about just going into Q&A?
Anybody got a Q that Mike can A?
I have a quick one. From a security perspective what’s your favorite cloud provider right now? Cloud services provider?
AWS by far. So I’ll tell you about it. So one of the really nice things about AWS, so there’s a couple of things that come up when you talk about cloud providers within certain industries, right? So, if you think of like big older companies that are terrified of the cloud, and that’s like, as somebody who’s run security at a couple of spots now, like everyone for the most part is in AWS or GCP or maybe Azure, but AWS rules supreme. One of the big objections that a lot of… When you’re a startup and you’re selling to an enterprise, one of the big things that comes up all the time is people are always super freaked out for no reason about being in the cloud.
So one of the stories I always tell when I’m in a room and I’m trying to help the sales team at Particle, for example, deal with an enterprise and they’re freaked out by the idea of AWS, I say, there’s a couple of things I would tell him. First of all, I know exactly where all my assets are, what they are, because you can’t do anything in AWS without it being in the dashboard in the console, right, without being able to see it. And I said to one giant company before, “I bet you a million dollars…” Didn’t say that you. You could not accurately tell me where all your assets are. And they probably have three or four different asset tracking systems and they’re probably all different. And I know exactly where my compute power is. I know exactly how much it costs. I know where the code was. That one itself is pretty phenomenal.
And then the other thing as well is I always tell people that I’d much rather my data be stored in AWS with all the default settings on that give you security then say, for example, on somebody’s random server in some data center somewhere or under a desk. One time I found there was an employee of a government agency that was hosting his own web server or web hosting company under his desk at the government office. And I found that because I was doing a pentest and ran a scan and found a website for a bakery in the outside range. And I was like, “Why are you hosting a bakery website here?” And they’re like, “Why not?” And I’m like, “Aah.” And then we traced it through. And it [inaudible] guys that. So just like the asset management side of things is, you need visibility to be able to protect stuff. How can you protect things if you don’t know where they are? That in itself is pretty much the biggest and best thing for the cloud.
And then the other thing is, with AWS specifically they’re just further along in terms of security tools and monitoring that you can enable. So for example, they have GuardDuty which is their inbuilt intrusion detection system.
It’s one click and you get coverage that companies will spend millions of dollars and years and years trying to get at and you can just get it in one place. So yes, AWS by far and away.
Oh, I got a Q. What’s GuardDuty?
GuardDuty, it’s like a intrusion detection system. So like what happens is so in AWS you have the concept of virtual private clouds, which are basically like networks within your account and essentially takes every traffic log or every time something flows across the three VPCs, every IP address, it will take a look at and it will flag if there’s any unusual or malicious activity associated with it. And they’ve recently started giving the same, but with S3 which is less storage solution logs as well. And one of the big horror stories that you hear about AWS all the time is how people accidentally expose S3 buckets to the internet, right, and then random punks can just show up and stop pilfering for them. The new feature in GuardDuty basically flags when that starts to happen. So if you have accidentally exposed an S3 bucket, which it’s getting very hard to do by the way, because they now also flag it with giant orange banners and things like that whenever you do it. So whenever that happens, it will trigger a new alert as well.
Cool. To be honest, I’m always skeptical of intrusion detection systems may seem to just be really difficult and expensive to deal with constantly throwing false positives at you and it really wears you down. So when a real event gets detected, there’s a chance that it’s just going to get lost in all the noise and who knows about all the false negatives, right. But you’re shaking your head yes. Does GuardDuty… Is it any better than the average? I mean, just in terms of its ability to actually do its job?
Yeah. At Particle… And to give you an idea of size, we have about… So Particle provides hardware that looks like this or like this, and we provide the connectivity and the cloud services around that as well. And so we have about a hundred thousand devices online every day, about 200,000 platform users. And so that’s quite a fair amount of connectivity and we get probably six or seven GuardDuty alerts a day, which is nothing really. To give you some comparison, right, say who is in this space?There’s no shortage of intrusion detection systems. You got open-source so close to us, but some of the most expensive intrusion detection systems out there, [inaudible] so let’s think, so FireEye, right. That’s a huge name in the world of intrusion detection. And when it came out, it was like, the latest and greatest in detecting what’s going on.
And obviously FireEye or Mandiant, which is a big consulting company as well. And so they are like a big cyber security player I feel like. And we evaluate FireEye at Concur and it had problems. And in order to deploy FireEye in every office location at Concur, which is what we needed to do because every Concur office had its own internet connection, so it needed every edge. You needed an IDS for us to be covered. We got a quote for $1.5 million, which was probably twice my budget at Concur. So I was like, “I can’t obviously do that.” So I did the same thing… We did like a proof of concept in a couple of offices there and then I went out and I bought… Obviously when they got that quote, I just laughed at them. And then I went out and bought these tiny little HP microservers that were 400 bucks each for like a, say two gigabyte server with three NECS in it. So I could just build my own IDS for 500 bucks and that’s what we did.
We ended up using Suricata, which is an open source idea and got pretty much the same information that we would have got out of FireEye, but obviously for much cheaper. And it worked very, very similarly. FireEye was probably just a little slicker on the user in space and things like that. And now the new FireEye, I purchased [inaudible] this before it gets is a company called Darktrace.
So Darktrace is a huge cyber security company out of the UK and they make a product that is very similar to FireEye and they do the whole thing of sensors and network intrusion detection and the like and things like that. But what they’ve done is they’ve done it in a way that it’s designed to appeal primarily to executives. And so literally they hired a person from [LucasArt] to design the user interface. So the user interface is like a computer game. It’s like this giant map of lasers shooting off all over the place. So for in an actual security operation center where you want to be getting as much information as quickly as possible in as few screens as possible, it’s complete garbage and no one will ever look at it and they’re very, very hushed about exactly how it works. And yeah, it’s like the perfect example of a tool that’s designed to look fancy and not have to do much.
So yeah, GuardDuty is the opposite of that. It’s very easy to deploy and very accurate, reasonable. And if you think about the volume of data, the Amazon clicks now in terms of like web traffic, as you know, probably the biggest host on the face of the planet, like they can use that to train their models pretty accurately. And it’s fairly decent. Akamai’s to be vendor neutral, Akamai’s a content delivery network and they do something very similar to GuardDuty. And again, you get better quality results based on the fact that they can see more what’s going through the internet and they can make better decisions.
Hey, Mike, a question for you. So for aspiring cybersecurity students, what would you tell them because cloud is a thing these days. What’s the best way to quote learn cloud? Is it hands on? Watching videos? What recommendation would you give someone who’s just coming in?
They all have, every cloud provider, Azure GCP, AWS all have free tiers, right? I mean, everybody learns differently, but what I would say is, there probably isn’t any substitutions just going in there and spinning up a free tier account. You have to give your credit card, but you can set emergency alerts and limits and things like that to make sure you don’t exceed the free tier, but you just want to make sure that you just go in there and explore the services and play around. The really cool thing about AWS is it looks the same whether you’re sat at home, just tinkering in it, or if you’re running a multi-million dollar company, right. They all pretty much look the same, it’s just scaled out, but different [inaudible] can go from spending $0 a day to spending a million dollars a day very easily, if you want to. Again, that’s why you put limits on spending.
So yeah, I would say just go tinker. You can try I think pretty much every service in AWS free for 30 days anyway, regardless of volume of events. So that’s what I’d do. And one thing that I’d say is, I’ve seen security vendors and people like that try and say that the network is dead and network security skills and things like that are dying out because everything’s in the cloud and everything is end point centric, now, especially now that we’re in a world where most of us will be working remotely for a pretty long time. And people saying that traditional network security skills aren’t as important anymore. But if you look inside an AWS cloud or inside a home network, there’s still a network. Everything still has to talk across the traditional TCP IP network. Still going to be intrusions and things like that, that show up on those networks. So understanding those fundamentals, even though they might exist within a cloud rather than in a traditional data center, for example, for most people, still super, super important and super, super valuable as well.
Oh, I’d happen to agree. I think that with the rise in home users, we’re seeing a lot more cyber incidents and attacks against the home user. There’s a lot of data that people are now pulling down from their cloud and manipulating directly on their machines on a non corporate network. And we’re also seeing, I mean, this last week, looking at all the stories of just as well, all the cyber ransom. The CryptoLockers have just exploded this. I mean, there’s a U.S. healthcare system that’s the largest in the U.S. just got shut down because of ransomware. So I think that to say that this is diminishing, I think it’s somebody’s buying too much into the cloud marketing platform.
Exactly. Yeah. I remember very early on, one of my favorite tests I ever did, there was a company like a defense contractor in the UK that they were experimenting with more remote work essentially because there was some reason. They were running out a room, I think was the primary reason in and there still is. So they were experimenting in making some roles remote, and this was like 10 years ago. So it wasn’t all that common in the UK. When you think about the UK compared to the states, it’s a good rule of thumb they’re about 10 years behind you. So they wanted somebody to go out and attempt to basically go sit outside an employee’s house and see what they could do while that person was connected to the network.
And I went to the first. They gave me like three addresses and I went to the first one, couldn’t do anything. Went to the second one, couldn’t do anything. So I tried the third one, and I was like, “Bingo.” It was a web network on web wireless networks. So super easy to crack into. And then I actually found out that they had not set up… They were supposed to use like an issued laptop for doing VPN stuff, but they’d actually installed it on their personal machine because they enjoyed having the larger monitor and keyboard and everything that they had with their desktop.
And so they had a very old desktop machine running a VPN into a fairly secretive defense contractor and the defense contractor couldn’t detect that because they just spun up this new VPN. So for them, it was just like a connected clients. So yeah, I was able to jump on the network and then find that machine because it was a, what’s called a split tunnel VPN. So you have half the traffic goes down to the tunnel to the employer and then the other half can still flow around freely on the local network.
It was pretty trivial at that point to pivot in and go explore the defense contractors network. So I imagine that there’s a bunch of very similar things happening out there in the world. This is the overlap in this space. I actually just got given… IT is one of my responsibilities at Particle because it’s now fully remote. It makes no sense to have an IT department in… We have one office, kind of. It makes no sense now. It’s more about just making sure that we issue endpoints with the right controls on them, that things like I just described don’t happen. So yeah, that’s one of my new things that I’ve been asked to take over with the help of some people, thankfully.
Can I ask you another question because I think what I see a lot of aspiring security students, there’s always the mythical, “I want to be at Cisco one day,” right. A lot of people, they want to become the Cisco because that’s the chief, right, and that’s the big bucks. So you’ve been on both sides, right? You’ve been the individual contributor on the engineer side, the pentester side, you’ve also seen the Cisco side. Can you talk about the advantages and disadvantages of both and where do you prefer to be? I know you said generalist, which is where I love to live as well. Can you talk about that a little bit.
It depends on the company. So I like what I do now because I am a CSO and I can be hands-on and I can basically… I like doing some of the business ey type things, but I also don’t ever want to lose touch with the technical stuff because I really enjoy it. And I’m fortunate in that I’ve landed in a place where I can be a bit of both and it’s not that big of a deal. In some companies, the CSO role is 100% business focused and you might have people that are very good at it in that position because… Well, so some people, I came from the technical side, right. So I’m what I classify as like a technical [inaudible]. Now I understand what engineers are talking about and I understand some of the business stuff and can help tie those things together.
There are some people that grow up completely on the business side and have no technical context with what’s going on. And that, I don’t think works in every place, but it works for some companies. And then, companies where it doesn’t really matter how you get to be a CSO, you just get there. And then you’re the first on the chopping block when something goes horrifically wrong, or you’re the first to get thrown in front of a customer in the event of [inaudible] happening. And that kind of sucks. So I think it completely depends on the environment and the company. For me, like, yeah, like I say, I have that good balance, so I’m fine. I would say, if you go through your career and you find… You’ve got to try everything once I think to find out what you really like.
I would say, don’t rush into it, just for the title. I made that mistake a little bit when I went to my first place where I became a CSO. I was desperate to get that job because I’d been trying for years and I really wanted to just give it a go. And so I rushed into it a bit and it didn’t really work out. And I was fortunate enough that I was able to land on my feet after it not working out, but don’t chase the title. Do what makes you happy. There’s no rush, right?
The [inaudible] is long and in the end, it’s only with yourself. And being a CSO, it’s great when it works out in the right environment, but it can be extremely stressful as well if you’re not in the right environment and it really, that stress, not to put anybody off, but for me, I’ve been in some pretty stressful situations in intelligence. I also have a pilot’s license.
So I have flown planes with people in them and had been in charge of planes and stuff hasn’t always gone well and I had to deal with it. So I’m pretty good at stressful situations. But it actually affected me physically when I was super stressed in the CSO roles that I didn’t enjoy it. So it really can take its toll and it’s worth knowing exactly what you’re getting into and getting an exact description of what the role entails before you commit to being a CSO or anything like that. So take time, make the good decisions, try everything once. But it all depends on the environment you’re in and make sure that you’re in a supportive environment if you’re going to go into a more senior role.
And questions to ask if you are going into a senior role as a CSO is, “What’s my budget,” and then, “No, seriously, what’s my budget?” Because they always give you a higher number than you actually going to get. And then, “What’s the size of my team. How many heads do I …? Things like that and that’ll give you a good understanding of exactly what you’re getting into and how much support you really have. Yeah. Sorry. I kind of went on a bit there, but-
No, I think I agree with you and I’d love to hear Kip’s comments as well, but oftentimes, you don’t want to be, as you know, you don’t want to be a glorified security engineer in an assistant role because a lot of people love the engineering side and they think [crosstalk] make the big bucks have to go into assistant role and then they hate it. Right? And oftentimes you do lose a lot of that hands-on type of work and feel free to chime in Kip. What have you seen? I mean, you should be doing less firewall implementation and intrusion detection type stuff as a system, right.
Well, I think it depends on the size of the company that you’re at. If you’re in a medium, if you’re in the low end of the medium sized company to a small sized company, you’re going to do everything. You’re going to scrub viruses off of PCs. And you’re going to meet with potential investors, and you’re going to fill out 500 questions, surveys from potential customers. And you might not even get that title, but that’s what you’re going to do. You’re going to do everything. Now, you get into say a $500 million a year sized company or larger then you should have a team. But I think once you get a team, once you have somebody who’s working for you, that is when you have to be a different person at work, I believe, because as soon as you have somebody that you’re delegating tasks to and they’re relying on you to write their performance review, advocate for them to get a salary increase once a year or whatever the cycle is, I think it’s irresponsible to continue to see yourself as strictly as a technical person.
You’re doing a great disservice to the person who is working for you, reporting to you. And so at that point, you really have to focus on what’s best for their development. And if that means that you should be training them to do the technical work that you enjoy, then that’s what you need to do. And you need to be willing to put some things down that you might have a lot of passion for, but if that’s what your person needs, if that’s what the people on your team need, then you need to do that and you need to be selfless and enough to do that. And I see people struggle with this, especially people would come up from the technical ranks I did. And that was a struggle for me. So I’m only speaking from experience.
Yeah. It’s very hard how that goes sometimes, right? For me, even at Concur, when there was an incident kicking off or something like that and I was a manager level there. So I had a team of engineers and they were perfectly capable of handling whatever was kicking off. But I’d still be hanging over the shoulder, like, “Hmm. Would I be doing that?” And I have to physically back off and go do something else to make sure I wasn’t freaking people out.
Yeah. Because you’re not trying to make people nervous and uncomfortable, but that’s what you’re doing, is making them nervous and uncomfortable. They don’t like it. And they’re going to start to associate that feeling with you whenever you come around and that’s going to set everybody up for bad. Yes.
Yeah. It’s definitely hard to delegate for a lot of people. So yeah, that’s something I’ve had to make a very deliberate effort to do is, hire people that can deal with ambiguity and just let them get on with it and it will work out and you can always step in if you need to. But yeah, it’s a skill. Learning to delegate is a skill that’s underappreciated, I’d say.
Yeah, definitely. So I-
When I hired teams in the past, I’ve always told them you’re professionals and I hired you for a reason. You come to me. There is no stupid questions. There’s no dumb questions. Come and ask me if you need anything, I’m here for you, but I’m not going to jump in and do your job for you. I’m not going to step on your toes. I’m not here to micromanage. I’m here to oversee the project here. It gets a lot more… But it definitely gets a better response out of the team, for sure, I would say. It’s that same thing.
I’ve spent years coming up through the technology side, in audio, visual and left for live events. And when I got to management, it was so hard to have to step back and not touch anything. I’m just like, I have to sit here and watch… Have a 360 view of the show, of a $2 million program for some large Fortune 500 company and you want to jump in and you want to put your hands back on it. And you’re like, “I can’t, I can’t do that right now. I can’t get involved. I need to trust the team that I hired.”
Yeah. And the other thing I’d say that, yeah. The other thing is like a big problem for me anyway, to overcome is when you’re an engineer and you’re hands-on, or you’re doing pentesting or whatever, you have a very specific work product that comes out of every day, right? So, “I’m going to write this code and ship this feature into the Sam,” or, “I’m going to set up this alert today,” or, “I’m going to complete this pentest.” So you have very specific goals, right, and there’s always some tangible output from the end of your day. And the challenge that you have in management and leadership roles is a lot of the times of day can just be spent.
I spent a day two weeks ago in 12 hours of meetings talking about things. And you feel guilty sometimes. Like, “Did I actually do anything today? Did I actually achieve anything? Or was I just talking the whole [inaudible]?” And it’s like, “Well, that’s the whole part of it. And it’s important too,” but yeah, you have to get into that mindset and you’re not going to have as much tangible stuff at the end of every day that you can lean on and say, “I did that today.”
Fortunately, right now I’m going through an audit as part of just normal stuff that we do. So that’s given me nice goals that I can set myself, giving me some of that good feeling about being able to achieve very specific things. But you don’t always get that the more senior you become.
I was wondering if I could ask you a question about your forensics background.
So I’m in a cybersecurity program right now, educational program. I’m starting to lean very heavily towards forensics. Two questions, really one is there anything that you wish somebody had told you about it before you got into it and two, how much of your pentesting skills dovetail with forensics work?
So on the first one, what do I wish people would have told me? I wish they would’ve told me that it involves real people and human emotions and things like that. One of the stories that I often… How is, I had to go image a laptop. So the story was that… This is in the forensics area somewhere. But the story was the, a person was being terminated from their job and they previously worked in the finance department at a government agency. And so what they decided to do for some reason was email a copy of the entire employee roster with account numbers, national insurance numbers, which are the equivalent of socials in the UK, and send them all out to sent… She sent them to her personal email account, right, essentially. And that got caught after the fact. So the file had been sent out and the forensics involved was basically, “Hey…” Well the story was, they said, “Oh, well, it was an accident. I accidentally selected the wrong file and emailed it to myself,” which no one believed.
But the story was this person was due a pretty decent payoff severance package for leaving that role. And so in order to take it, they put a hold on that severance package and said, “Hey, in order to get this, you have to agree to have forensics done on your laptop to make sure that you didn’t keep this file around and you deleted it like you said you did.”
So what happened was I had to go image this machine and it was a 750 gigabyte Dell machine. And this was like 10 years ago. So it was fairly slow to image and I had to go do it in front of that person. And as it turned out randomly, their 17 year old kid who came along to the lawyers office for whatever reason and so I sat there and I was just imaging this machine. And it was very awkward. And that’s probably not the only situation I’ve been aware of. There’s like actual real people involved. And it’s getting awkward now. So that’s something to bear in mind.
And then the other question about pentesting involved. So yeah, one often leads to the other, I’d say. So the pentesting… So let’s think about this. I’m trying to say a good example of where they intersected.
So, yeah, it was a pentest that led to the discovery of the dodgy admin control panel thing that was exposed to the internet. That actually started out though as a pentest, but it quickly evolved into a forensics exercise when we found out that it had been like that for a year. And at that point you can’t really trust anything in the active directory that that thing was in front off, because who knows who’s been in there and doesn’t work? So then yeah, you tend to find things in pentesting that may need forensics, or you may find things in forensics that may need pentesting to better understand the vector for how things got in there. So they do intertwine very closely. And it’s good to have a bit of an understanding of both, for sure.
Fantastic. Thank you. That was excellent.
We do have a question in the chat room, if it’s okay for an ask. And it’s how do you prepare to become a CSO, especially if you’re coming in from a GRC background?
I would say, [Len], if you’re coming from a GSE background, take the time to learn as much about the technological side of things as possible, just to understand when engineers and developers come to you with technical problems, you don’t have to be the smartest technologists in the room to understand them. You just have to be able to reassure people that you understand enough to be able to action the concern appropriately. And one of the big challenges, especially in tech companies and… I’m in the Seattle area as well. And it’s like, obviously a lot of tech companies up here and one of the things is that everybody has a right to be given the opportunity to take on that job no matter how they got there.
But sometimes the more technically minded folks can believe that you have to come from a technical background. So you have to just understand and emphasize with whatever they’re trying to raise as best as you can and make sure you know how to ask the right questions to develop an understanding of any kind of technical thing, because obviously it’s a very technical role, but it doesn’t have to be the whole time. So, given what you can to broaden your understanding of the point why you can ask the right questions and work with people of varying technical skill levels is super important.
I had a question about aspiring or junior members who joined your team when you were a hiring manager, in terms of what personality traits did you find or stories that you have, where they succeeded? What was some of your best employee members?
Yeah, so actually one of my best hire’s was a lady called Darcy and I worked with her at three different companies. I hired her at each one and she didn’t have a huge technical background. She came from law enforcement. She worked in the State Gaming Commission or something like that, and did investigations for them. And she also came from a CPA and audit background, but she wanted to get into the technical side of things. And just her ability to understand some of the business-ey side of things, understand some of the financial side of things. So we first met at Concur. So that was an expense platform. So there was obviously more financial relevance there, but just the ability to never stop unrelentingly digging into things was phenomenal. And that’s why she’s so good. She would just keep drilling into things and is not afraid to ask the hard questions and never takes things at face value or has always dove into things. And that kind of, for me anyway, at the time, that kind of complimented my more hands-on technical skills and that was why we paired together so well.
So that’s a success story. I’ve also had some not so successful stories about people I’ve hired. So in the interest of balance, I hired probably the best technical person I’ve ever met and I ended up having to let them go because they came with a very serious bunch of technical skills, but absolutely zero desire or ability to want to hang out as part of the team or be part of the team. So they thought that they had special… Because they were so advanced technically, they felt they could excuse themselves from doing some of the daily stuff that everybody on the team did, no matter who they are.
And so that one didn’t work out and it was annoying because I really needed that technical skill, but I also knew that the house as a team was more important than having one person that was really good. So I actually wrote in security operations and practice, there’s a whole, that’s [inaudible] those, the last book I wrote, there’s a whole section in there about personality types and how they fit on different teams. I’m trying to think of some of the types, but it was things like accountant type, business person, technologists, developer. There’s a role for all these different skillsets to play in a security team. And I think one of the lines I put in there was like, “People often say that there’s a skill shortage in cybersecurity and I don’t believe that’s true. I think there is a shortage of people with the vision to take people with different backgrounds and put them together and apply them in a cybersecurity context. But I don’t think there’s a shortage of actual people capable of doing the work.
So yeah, it takes all sorts of personalities to mesh together and make a successful team. And it takes only one of those people to have a really good skill set and not fit in with the team and think they’re slightly different for some reason to screw it up.
Thank you. I had something related to that in terms of what stands out about a person way before you hire them, right? How did they come into your radar? Was it through personal connections mostly that you hired through… Again for junior roles? Or was it something in their resume stood out, something of their work stood out?
I would say connections never hurts. I think I’d say that I’ve hired through personal connections and basically through somebody who’s seen a person work a fairly decent amount, probably about 50% of the time. But there comes a point when that doesn’t always work anymore because you become friends with the person and you don’t always want to be friends with new teams, which is a weird thing to say, but it becomes harder to manage them professionally if you’re really good friends, because if you want to make a change or if something’s not working out, then it becomes harder to raise, right. So I’m always a bit cautious about that. Also, I’m a bit like Ron Swanson and having three friends is sufficient. But so I’d say half with personal connections to just people that you’ve seen work before or other people on the team have.
And then I always, like… I look at resumes and I look at like… Especially in junior roles, what I look for side projects and research and things like that that people have done. And you see in usually the junior roles, people are very keen to show you either their technical skills or their investigative skills. And they dread the actual talking is the view part of an interview. So it’s usually a pretty good indicator of what someone’s going to be like. So what I did once, I was hiring for a pentest lead position. And so it made sense, right? I had like a bunch of vulnerable web apps that I’d written. So I was actually teaching a course at UDaB about pentesting. And so I had all these ready to roll.
So I was just like, “Doing these interviews, I’ll just throw that up and see if anybody wants to play on it, well, instead of actually answering questions.” And four out of the five candidates for a pentest lead position were just like, “Aah, no, no, I don’t want you to interview for that.” And then there was the one person was like, “Oh, sweet, this is way better than answering questions.” So they just got stuck in there and they didn’t break into the whole application, but they just showed me how they were thinking about it and things like that. So I think understanding that people interview in different ways and particularly in more junior roles, people just like the opportunity to be able to show you the technical skills and things that they have versus, conversations and answering questions about theoretical things.
So I always try and do that. One other interview that I did, I hid a… I didn’t mention it. I just hid a, what’s called a [inaudible] plug. It was a pentesting Dropbox, and I just hid it in the room, like in plain sight. It just looks like a power supply, but it’s actually got a raspberry PI in it and is used to give you a [inaudible] on your network and I hid it there I just wanted to see if anybody said anything. And I was in [inaudible] and yeah, sure enough, five candidates, one spotted it right away. Was like, “Why do you have a palm plug in here? Do you know that’s in here?” And I’m like, “Yeah, I see.” So just little things like that, that help you see through things.
Another story I’ll tell very quickly is the reason you can trust resumes, right, is like I got this awesome resume. Brought the candidate in and this person had a bunch of different experiences in all these different things. And I was like, “Holy [inaudible], this is really awesome. I need to get this person in straight away. Got them in. And I said, “Oh,” because in the [inaudible] you get a lot of forensics that [inaudible] and then person said, “Well, what happened was is whenever forensics needs to be done, I would bring in a consultant.” And I said, “Okay, well, it says you did pentests in here. Did you pentest it?” “Well, what happened was is whenever I need to do pentesting I brought in a consultant.” I was like, “Oh, so have you done any actual hands-on forensics or pentesting?” “No.” I was like, “When I look at my resume, it doesn’t say plumber, electrician because whenever I need those things, I bring in a consultant who is a plumber and an electrician.”
So you would have to be a bit careful with resumes because obviously some people like to make things up to a different degree than others. So that’s why I always liked to go face to face and just see how people react when they’re given a technical challenge or how I feel like they fit into a team. I think that’s more important than just doing it on resumes alone, but you have to start at resumes. So I always look for things like five projects and it adds to the experiences rather than things people say they do and as a job, for sure.
Cyber Risk Opportunities
Kip Boyle serves as virtual chief information security officer for many customers, including a professional sports team and fast-growing FinTech and AdTech companies. Over the years, Kip has built teams by interviewing hundreds of cybersecurity professionals. And now, he’s sharing his insider’s perspective with you!
Dion Training Solutions
Jason Dion is the lead instructor at Dion Training Solutions. Jason has been the Director of a Network and Security Operations Center and an Information Systems Officer for large organizations around the globe. He is an experienced hiring manager in the government and defense sectors.