The Air Force Academy Gets Lean

I can always tell when one of my students has been in the military. They’re focused, they’re world-wise past their years, and they don’t break a sweat in the fast pace and chaotic nature of the class and entrepreneurship. Todd Branchflower took my Lean LaunchPad class having been entrepreneurial enough to convince the Air Force send him to Stanford to get his graduate engineering degree.

In class I teased Todd that while the Navy had me present my Secret History of Silicon Valley talk in front of 4,000 cadets at the Naval Post Graduate School, I had yet to hear from the Air Force Academy.  He promised that one day he would fix that.

True to his word, fast-forward three years and Todd is now Captain Todd Branchflower, teaching computer engineering at the Air Force Academy.  He extended an invitation to me to come out to the Air Force Academy to address the cadets and meet the faculty. Besides the talk I brainstormed with Todd and other faculty on how to integrate the Lean LaunchPad into the Air Force Academy Capstone engineering class (a Capstone class puts together all the pieces that a students has learned in his or her major.)

Here’s Todd’s story of how we got there and progress to date.

——-

Not That Long Ago
In 2007, I graduated United States Air Force Academy as a computer engineer and entered the Air Force’s acquisition corps, excited and confident about my ability to bring technology to bear for our airmen.

Graduation day with classmate Joseph Helton (right), killed in action in Iraq in 2009

Graduation day with classmate Joseph Helton (right), killed in action in Iraq in 2009

And I couldn’t have been put in a better place: testing the Air Force’s newest network security acquisitions. I was their technical man on the inside – making sure big defense contractors delivered on their promises. We were modernizing datacenters, buying vulnerability-scanning software, and adding intrusion detection appliances – all things typical of anyone running an enterprise-scale network..46th test sqd

I was in the thick of it – chairing telecons, tracking action items, and drafting test plans. I could recite requirements and concepts of operations from memory. I was jetsetting to team meetings and conferences across the country. I was busy.

Sure, I wasn’t working very closely with the airmen who were going to use the equipment.  But they called into the weekly telecons, right? And they were the ones who had given the program office the requirements from the outset. (Well, their bosses had.) And I’d distilled those requirements into system characteristics we could measure. Well, more measurable versions of the original requirements. And meeting the requirements was the most important thing, right?

Doing it Wrong
Here’s what I learned: I was doing it wrong. The way our process worked, customers were just a stakeholder that provided input – not drivers of the process. That meant that program offices were only accountable to a list of requirements, which were locked early. Success only consisted of passing tests against these requirements, not delighting our airmen. I began to wonder – how could we learn about user needs earlier?  How could we deliver them solutions more quickly?  More cheaply?

It was only after returning to Stanford and taking the Lean Launchpad class that I became convinced that a radically different, customer-centric approach was the solution. I returned to the Air Force Academy as an instructor in the Electrical and Computer Engineering Department, intent on spreading the gospel of Customer Development and Lean.academy ee

Our existing Capstone senior engineering design course followed the defense acquisition process; the focus of defense acquisition is to “nail down requirements” early and manage customer expectations to “avoid requirements creep”. I saw this as counter to the joint, iterative discovery process between entrepreneurs and customers I had experienced on my Lean Launchpad team.

I kept in touch with Steve as I started teaching. We discussed how the Lean Launchpad approach might find a place in our curriculum, and how it might be adapted to fit the unique Air Force Academy / military environment. We grew excited about how showing success here might prove a good model for how it could be done in the broader Air Force; how exposing future officers to the Lean philosophy might bring about change from within.

So when I invited Steve out to the Air Force Academy to speak last spring, there was more at stake than the talk.  We set up a meeting with our department head, Col Jeff Butler, and Capstone course director, LtCol Charlie Gaona, to pitch the idea.  They shared our enthusiasm about the impact it could have on our future design projects and how it might bring a change in perspective to our acquisition corps. They gave the go-ahead to send a pilot team through the program in the Fall semester, with the potential for it to be applied across the entire course if we delivered results.

I found a willing co-conspirator in Capt Ryan Silva, a star instructor who mentors a project named Neumimic, using technology to aid in the rehabilitation of patients with chronic loss of limb motion.  In the first year, they had developed a proof of concept around the Xbox Kinect – and Ryan had high hopes for the future. But he found some elements of the traditional systems engineering process cumbersome and frustrating to cadets. Ryan signed on to lead our test class.

V-Model of Systems Engineering
The current Capstone class follows the V-Model of Systems Engineering, with teams creating a detailed system design throughout the Fall semester and building their design in the Spring.

Vmodel

There are a series of formal reviews throughout the two semesters, in line with the Air Force acquisitions process.  Requirements and a concept of operations are presented at the first, the System Requirements Review.  Cadets receive instruction on the process in about a quarter of the course lessons.

What we decided to do instead was have semi-weekly informal reviews Lean Launchpad style, focusing on product hypotheses, customer interactions, learning, and validation / refinement.  We emphasize customer interaction via “getting out of the building” and rapid iteration through “cheap hacks”.  We’ve removed most of the structure and firm requirements from the original course in favor of a “whatever it takes” philosophy.  Instruction is presented in tandem with the reviews, focusing on areas we see as problematic.

Last year’s team meeting with Dr. Glen House at Penrose-St. Francis Hospital

Last year’s team meeting with Dr. Glen House at Penrose-St. Francis Hospital

Back to the Present
We’re about a quarter of the way through the fall semester. Team Neumimic consists of nine sharp cadets across multiple academic disciplines. Based on initial customer interactions, they divided themselves into two complementary but standalone teams. One will focus on design, execution, and measurement of therapy sessions – building on the original Xbox Kinect work.  The other will work on adjustable restriction of patient motion – forcing patients to use the proper muscles for each movement.

Here’s Ryan on the impact of the process change:

“Last year the team found themselves handcuffed to a process that required a 100% design solution on paper before we could even think about touching hardware…crazy right?! We spent the entire first semester nailing down requirements for a system that was supposed to meet the needs of stroke and traumatic brain injury patients as prescribed by their occupational therapists. For five months we slogged our way through the process emerged with a complete design for our system, custom-built to meet the needs of patients and doctors alike. Our design was flawless. We had nuts-and-bolts details all the way down to the schematic level. We were ready to build! The fact that we had yet to even see a patient or spend any real time with an occupational therapist had not even registered to us as a problem, until we were invited to watch a therapy session.

Our entire team walked out of the hospital ashen-faced and silent. We knew we had just wasted half the course designing a system that wouldn’t work. We were back to square one. The remainder of the course was spent in a frenzy of phone calls with doctors and therapists paired with many design reviews, but this time with our customers in the room. We were able to iterate a few solutions before we ran out of time, but the customers were thrilled with what they saw. I could only imagine what we could have accomplished if we didn’t waste the first half of the course on a solution that ultimately wasn’t what the customers wanted. I was fired up when Todd approached me with his idea to fundamentally change the way we did business.

So far the results have been incredible compared to last year. The team has learned more about the problem in a month than last year’s team learned in an entire semester. I’m not saying this year’s cadets are any more capable than last year’s; just that I believe this year’s team has been given a better chance to succeed.  They’re freed of a lot of stifling overhead and are embracing a process where requirements are derived from those who will actually use the system…imagine that! I’m excited to see what the team does with their remaining eight months.”

Current team members observing Dr. House conduct a therapy session

Current team members observing Dr. House conduct a therapy session

But we have experienced challenges in implementing this approach. Here’s what we’ve noticed so far:

In typical Lean Launchpad classes, students apply as teams with their own idea.  There’s also the potential for teams to pursue the opportunity beyond the class if they’re successful. In our Capstone, projects are predetermined and cadets are assigned based on preference and skill set.  Cadets will graduate and be commissioned as officers, doing various jobs throughout the Air Force. It’s highly unlikely they’ll be able to continue their project. These factors might make the initial motivation of our team less than that of other Lean Launchpad teams.  We found that early interactions with customers excited about their work went a long way to remedy this.

We’re offering cadets much less structure than they’re used to. Some cadets are uncomfortable with the ambiguity of the requirements (“What are you looking for?  What do I have to do to get an A?”).  I’d imagine this is typical of most high-performing students.

We’re trusting cadets with more freedom and less oversight than they’re used to.  There’s the potential for our trust to be abused.  I’m hopeful that our cadets rise to this challenge.  I think they’ll feel ownership of the project and empowerment, rather than see an opportunity to shirk responsibilities.

Since this course is a senior design experience, cadets expect to be “using their major”.  There’s the tendency for some to sit on the sideline if the pressing work isn’t directly related to their area of expertise.  It has taken some prodding for cadets to embrace the “hustler” mindset – to take any job necessary to move the team forward.

These are challenges we can overcome.  I know we’re moving in the right direction.  I know we have the right team and project to be successful.  I know our cadets will make us proud.

Up the hill!

Listen to post here
Download the post here

Why Real Learning is Outside the Building, Not Demo Day

Over the last three years our Lean LaunchPadNSF Innovation Corps classes have been teaching hundreds of entrepreneurial teams a year how to build their startups by getting out of the building and testing their hypotheses behind their business model.  While our teams have mentors, socialize a lot and give great demos, the goal of our class final presentations is “Lessons Learned”  – about product/market fit, pricing, acquisition/activation costs, pricing, partners, etc.  We think teaching teams a formal methodology around the Lean Framework (Business Model design, Customer Development and Agile Engineering) is a natural evolution of how successful incubators/accelerators will build startups.

Here’s the story of one such team; Jonathan Wylie, Lakshmi Shivalingaiah and the Evoke team.

—–

Imagine if, in the course of ten rollercoaster weeks, your customer segment changed from executives on corporate campuses to moms on playgrounds, a tool that was just part of your product turned into the killer product, and the value of the problem you were solving went from number 47 to customers trying to give you money when you demo’d.  Here’s how that happened.

We came to the Lean LaunchPad class wanting to build a mobile/web research management system aimed at helping qualitative researchers better manage the media they captured in the field. We were ready to learn, but pretty confident we would end the journey in the same market space in which we started.  We had a killer team and all the right skillsets.  I was a consultant and ethnographer, another teammate was a market researcher, and two others had the software engineering skills to build what the market needed.  And what the market needed would, of course, be exactly what we had envisioned. After all, there must be a huge number of researchers struggling with the exact same problems we had, right?  Not quite…

Out of the Building
In the first 4 weeks, our team got out of the building and spoke with employees at 42 different companies.  We spoke with people at all levels, from front line user experience researchers at large tech firms to the CMO of a fortune 500 consumer goods company. Discover X WorkflowFrom the first 10 interviews, we learned that video is a big problem for researchers who use that medium.  It takes an average of 4 hours to mine every hour of video for the relevant 10 seconds of insight that matters.  Thus, we focused our early minimum viable product on helping researchers save money and time in finding insight in market research videos.

Wireframes
We built wireframes as a Minimum Viable Product to elicit feedback and began showing them to customers during our interviews.  At this point, things got real…and a bit ugly.  Given something tangible, customers were able to start gauging their willingness to use and pay.  Discover X wireframeTurns out, researchers were “just not that into us.”  We heard consistently that the product looked good and solved a problem, but it was not an important problem.  It was number 47 on their list, and there was no way they could justify paying to solve that problem.

First Pivot
As disappointing as this was, we dug deeper with our questioning.  To our surprise, customers started offering ideas on where there might be a true need; one of which was the legal market, specifically the deposition process. We thought this would be perfect for our product. There is a lot of video being recorded, and attorneys need to be able to pull out the insights quickly. After a solid week of speaking with lawyers and attending webinars on real-time deposition software, we had mapped both the technology and the buying relationships.  What we learned was that, we would just be an incremental feature to the incumbents and would need to integrate our solution with theirs. This, combined with regulation from the courts, a 2-year sales cycle, and the realization that e-discovery groups are not early adopters, made this an unattractive market.

Technology in search of a market
By this point, we were a technology in search of a market…not a good place to be.   The next customer segment we tried was startup founders.  After all, they are just like us – researching their markets and needing a way to share insights and keep their teams connected to customers. However, we found that most just assume that what they are building will have a market. The few who did get it felt uncomfortable using video during the interview process.

Pivot Two
While at times we felt like we wanted to give up, we began to hear a positive signal in the noise of all the customer rejections.  Evoke BrainstormingAt first it was faint.  While customers in all three markets were lukewarm for use at work, they got visibly excited telling us that it would definitely solve a problem at home. Say what??  They told us “too bad we weren’t making a consumer product so they could document their kids… they would pay a lot of money for that product.”  Whoah…were customers telling us we are a consumer product rather than B-to-B??

We settled on a small-scale experiment to test the consumer market. We decided to speak with 10 parents over the course of a week. If 5 had a similar problem, we would dive deeper. What we got was a landslide of interest.  All 10 parents had the problem.  Even more amazing to us, 9 of them liked our solution!

We learned that parents capture moments with their families to:

  1. remember and relive later
  2. share with those closest to them
  3. pass along a memoir to their kids

To our surprise, it turns out that none of these are being accomplished well with existing products, and parents are stressed because they feel like they are failing in an important responsibility.

Eureka!
Since that initial experiment in class, we’ve validated these findings (and many others) during over 200 hour-long interviews.

1st evoke wireframes

We even partnered with the university on a 112-person design workshop to learn more about how photos and videos fit into people’s lives.  It’s always an incredible experience to be invited into someone’s home to learn about how they capture their most precious family moments.  Sometimes, the learning is immediate and conclusive. Other times, we have to do multiple rounds before we arrive at an answer to an important question.

The result of all this effort is that we have found a large and underserved market in hidden in plain sight, right in the middle of an area that gets a lot of attention – photos and videos!

Lessons Learned
There’s no way we would have learned any of this unless we were out of the building and in the trenches, with parents over an extended period.

Knowing our customers and their problems first hand has given us a huge head start and a competitive advantage. Most entrepreneurs seem to just make this stuff up for a pitch deck or to please stakeholders, but the validated learning that we gained through these interviews and other methods of business model experimentation is not something that can be easily replicated.

As for our current status, we are building the product, continuing customer development, exploring and validating other aspects of our business model, and…oh yeah…hitting the pavement to raise our first round of funding!  If you want to talk to us about that, or if you know parents that we should be speaking to, please feel free to reach out.

For all the parents out there, relief (and much more) is on its way… http://www.evokeapp.com

Listen the post here
Download the audio here

%d bloggers like this: