Almost four years ago, we started our two year effort to write the go-to book for first time entrepreneurs. Our reasons were pretty straightforward – we couldn’t find any books available that answered the basic questions that we hear startup founders ask all the time. Other books seemed to be written more to serve what the authors had to sell – venture capital, legal services, financial advice or just their ego – than the founders’ needs. Being founders of many companies ourselves, we wanted to write a book from the entrepreneur’s side of the table. Thus came about the creation of The Startup Playbook.
If we had known what we were getting ourselves into, we may have passed on the effort. It took way longer than we expected. We, essentially, wrote the book three times. Some of the lengthy process was because we knew nothing about writing a book; some of it was that we were perfectionists in the getting down the information we thought new founders needed; and some was taking the time to get feedback from a large number of super helpful beta readers. Also, as we talk about in the book, it’s hard to make a success out of a side gig and the book was definitely a side gig for us.
Yesterday, exactly 18 months since we introduced The Startup Playbook, we “shipped” our 10,000th copy. As it turns, out, that’s the number of copies we set out as our goal. We thought that if we could impact that many entrepreneurs, the journey would have been worth it. We priced it aggressively so that cost wasn’t a barrier – we never even planned to make money on the book – and several accelerators have used it as part of their programs. We’ve learned a lot along the way and have gotten great feedback from or readers. The book was on Amazon bestseller lists multiple times and currently has a 4.9/5.0 rating.
Along the way, we’ve learned a lot. 10,000 was our initial goal, but that’s not where things end. We still talk with new entrepreneurs about the contents of the book and offer guidance along those lines as often as possible. Book sales continue, of course, and we really enjoy the fact that so many people have gotten so much out of it.
Thanks to everyone for all the support of the book. 10,000 is a great milestone, but we have so much more to do!
In The Startup Playbook, my co-author, Rajat Bhargava and I share the lessons, tips and even shortcuts we’ve used in starting, running, advising and investing in companies over several decades. These include tactics for refining your idea, team-building, raising money, developing a product or service, and, ultimately, how to execute.
With a foreword written by Über entrepreneur and venture capitalist, Brad Feld, the book addresses the specific issues and opportunities that founders face as they start and grow their businesses. And, because it’s a window into how founders think about their startup, The Startup Playbook is also helpful to everyone in the extended startup team.
A painful truth of entrepreneurship is that the vast majority of startups fail. There are many reasons for this, of course, but almost all of them come down to the simple fact that new entrepreneurs haven’t yet done and seen enough to have the wisdom required to avoid many mistakes, know what questions to ask and leverage new opportunities.
We share our experiences and what we’ve learned from them in a founder-to-founder discussion style. This isn’t a compendium of what one-hit-wonder founders, VCs and talking heads in the media want you to hear. It’s an in-the-trenches guidebook, written by serial entrepreneurs that is meant to shift the odds of success in your favor.
Even if you’re just dabbling with the idea of becoming a founder and starting this journey, The Startup Playbook can help you to determine whether it’s the right path for you.
I made my first angel investment in 1994. At the time, I had no basis for evaluating whether I was investing in a good idea, understanding how the deal was structured or even what the market the company was addressing looked like. It just seemed like a cool thing to do – great founders and a killer co-investor in Brad Feld. As it turns out, that first investment, net.Genesis, was a big success. While I’d like to think that there is some correlation between my choosing net.Genesis for my first investment and its success, I don’t think there is much. Mostly, I was lucky.
After dipping my toe into the angel waters the first time, I continued to invest in more new companies. I made 1-2 investments every year for a while, always going in relatively big in my first round and picking up my pro-rata in at least the subsequent round. For the most part, I still didn’t know what I was doing, but I was beginning to get a feel for things. Since I was only doing a couple of deals each year, I started doing more due diligence, but even this was limited as I had a day job running my own company.
Some companies I invested in succeeded, more failed. For the most part, though, I was doing pretty well, making more money than I was losing and making way more than I could have in the stock market or any other investment instrument I had access to. Still, no one was going to question my amateur status. I was choosing companies to invest in from a very small sample of those looking for money, I didn’t understand the subtleties of deal terms (as I often co-invested with VCs, this lead to problems), and I was still investing in companies addressing markets I didn’t have even a basic understanding of (I’m talking to you, life sciences and healthcare).
Eventually, I got better at judging companies, teams, ideas and markets and being able to change a sea of gray into black and white, especially with respect to the market part. I increased my pace of investing to move up to 5-7 deals/year, but started investing less with each company and not always reinvesting my pro-rata amount in subsequent rounds that I thought were too expensive.
Fast forward to today. While I’m still an amateur, I’ve now done over 60 investments. Not as many as some, and not even close to semi-pro and professional angels. Still, a reasonable number. I finally feel that I have enough data to try to extrapolate some trends and, perhaps, associate some causes and effects. I’m still learning, of course, and since I haven’t made a profession out of it, I still don’t always hold fast to my own observations/guidelines. But, for what they’re worth, I share them here.
The General Stuff
- You need to put a reasonable amount of money in play especially at the beginning. Nothing pays out for a long time and the feedback loop is huge. A small percentage of investments will return anything, so you have to place a lot of bets.
- You need to look at many deals before choosing which ones to invest in. It’s easy to get excited about the ones you see, but you have to ask yourself what you’re not seeing. There are a million little companies out there to invest in. What are the odds that the subset you are seeing are the best ones?
- Network, network, network – get close to as many other angels as possible. The vast majority of angels are very open and are happy to share deals with you, make introductions or give advice.
- Look at every company from every incubator/accelerator you can find. Meet with the companies if you can. That’s not to say that there are good companies at all incubators, although there are usually one or two in each session, but there are many in one place and they’ll give you a perspective on what’s going on, team dynamics and what others are looking to invest in. They are also good practice for your own discovery skills. Like Malcolm Gladwell says in Outliers, you need to do it 10,000 times to build an instinctive foundation for knowing what has the best odds of working.
- As in most things, luck remains a significant factor in successful investment returns. When I think I’m getting good at investing, I remind myself to look back and ask myself if the company succeeded because of factors I had predicted, or because of something else. Most often, success is greatly determined by the company being at the intersection of the right place and the right time. Often, it got there because of macro-economic changes that were out of my view and out of my control. Sure, picking the right team in a solid market influences arriving at this intersection, still, there is a lot of luck involved.
What I’ve Learned and My Current Thinking
- Almost any deal with a good return will take a long time to reach a liquidation event – 6-10 years. While there are exceptions, my home runs all took about a decade to return their first dollar.
- I used to think I could put a small amount of money into many companies each year and things would play out. I no longer believe that to be true. It’s a hits business. Even with a high-multiple exit, a small investment won’t provide a large enough absolute return to cover losses from failed investments and make a lot of money. Now, I invest more in the first round I participate in and almost always do my pro rata in subsequent rounds to maximize my ownership as the company grows.
- I try to be somewhat valuation agnostic. That is, the math on successful companies (assuming my home run theory is correct), shows the valuation at which one invests really doesn’t matter much. I try not to overpay of course – if I can’t acquire enough of the company to give me a good return when a logical exit takes place, I’d rather put my money at work somewhere else.
- There are red flags (real or potential problems) in all deals. I try not to let the existence of one scare me away. It’s the combinations of red flags that matter. If the CEO does not want to take money for fear of being diluted AND won’t create a board of directors, for example, I walk away.
- I have two fundamental requirements for an investment – the team has to listen well (they don’t have to ever take my advice, but they have to listen to it and consider it), and founders have to have a desire to build a killer, well-rounded team. In my experience, founders who don’t actively listen don’t succeed. Those without solid teams struggle and most eventually fade away.
- Ideas are fleeting and are almost never unique. Even if the company seems to have a distinct cut at something, someone else with a similar idea will be smarter, faster and have more capital. Speed is the key variable. If the team is willing to work their asses off and take enough money so that they always have fuel in the tank, then they have given themselves the best chance to succeed. This latter part – money – is where I often find problems. Sometimes, founders don’t want to take money for fear of dilution. Because they don’t have enough money, they move more slowly and are overtaken in the market.
- I hate convertible notes, but after years of therapy, I accept that they are here to stay. Still, they are loaded with pitfalls and I try to avoid them when I can reasonably do so.
- I only invest in stuff I know, at least broadly. Of course, it’s not possible to know the entire spectrum of products, services and markets I want to invest in, but there are always some I have a better feel for than others. I have lost a ton of money in healthcare and life sciences. I’ve invested in the cure for cancer 3 times. It just goes to show that I have no idea how that stuff works.
- Does the company have a board? Every company should have a board. I find that often, when startups don’t have a board, it’s because they are worried about loss of control and put that worry ahead of the value that close advisors can bring to the company. This is a huge red flag for me (see above).
- I don’t invest in a round when I think the company will struggle to create a significant step up in valuation before they need more money. This can happen because the valuation on the current round is too high or the company is not taking enough money to make significant progress before the money runs out. I just wait until the next round to invest at pretty much the same valuation with reduced risk.
- There are many angel investors who seek out teams in the very early, formative stages of building their companies. As you’d expect, this type of investment offers the potential of lower valuations, a much greater return and, of course, failure. I am not one of these investors. I’m interested in the follow-up investment right after this nascent round if the company makes it that far. Much less risk and less work as well as I tend to follow and not lead, but my experience is that I usually have to invest at 2X the valuation.
There are some hugely successful, high-profile angels who can offer more wisdom and informed investment criteria than me. I fully suspect that some very successful angel investors may even disagree with me on one or more points. That said, for those of you who are just getting into angel investing or are looking for some perspective on how others do it, I hope that this is valuable. As always, I’d love to hear what you think.
Not so long ago, when one invested in an early startup, it was almost always through the purchase of preferred stock or equity in the company. An exchange of cash for a small percentage of ownership. Neat and tidy, everything on the table, no loose ends. Then, over time, founders (with the help of incubators and accelerators) decided that these equity instruments were becoming too difficult and time consuming, especially with respect to the negotiation of the price of the round and, to some extent, the preferences required by some investors. The “simplified” instrument that most companies started to use was the convertible note.
Convertible notes quickly became de rigueur in the startup community. They are, ideally, debt instruments that offer the investor interest payments in exchange for cash invested in the company. On the successful achievement of certain milestones, the investment (plus the interest earned) is automatically converted into ownership in the company. The key milestone generally being the closing of a subsequent equity investment round. Allegedly, if the company fails to raise an equity round in a specified timeframe, the investor can get their money back. This is supposed to be the advantage for the investor but is, in fact, a useless provision. If the company can’t raise a round, they have more than likely spent all the cash they have leaving nothing in the till to return to investors.
Initially, convertible notes did make things simpler for the founders of the startup. They could kick the valuation discussion down the road and the paranoid founder could withhold any control and financial preferences from their early investors. Unfortunately, these early, simple convertible notes were minefields of problems for angel investors. Foremost among these:
- They created situations where investors might only get their money back even though there was a large exit for the founders. Think of an exit that happens before a conversion takes place.
- They did not specify any control provisions or establish a board (every company needs one). No oversight.
- They often didn’t recognize the level of risk taken by the early investors with an appropriate combination of discount and conversion restrictions at the next financing round.
- Because the investment is debt and not equity, they kept the investors from starting the clock on capital gains treatment for the investment until the conversion took place.
To address these problems, investors negotiated new provisions in the notes to patch the holes which, in turn, made the convertible note documents increasingly complex and ultimately created documents at least as complicated as the equity investment instruments that they replaced.
Now, instead of discussing the valuation of the company and specific preferences of the preferred stock in an equity round, the cap on the note (the maximum conversion price), the discount (the reduction in price upon conversion), and the term of the note (when and how conversion takes place) need to be negotiated (some good descriptions of these can be found here).
Convertible note documents have ballooned into multi-page, complex forms that are not only negotiated just like their equity-based brethren, but now carry legal costs that are similar as well, taking away much of the advantage that was sought when they were widely adopted in the first place. In fact, they don’t even succeed any longer at their primary goal of kicking the valuation discussion down the road since the negotiation of the cap on the note has replaced that pretty much 1:1.
The realization that convertible notes are no longer either cheap nor fast has brought about several openly available standardized equity financing instruments like Series Seed, Techstars Open Source Model and others that promise to make a priced round even cheaper and faster than a current, complex convertible note. In my experience, and sadly, these documents are not changing the landscape much.
OK, that’s my rant. For a while, I refused to invest in converts. I wasn’t the only investor around who felt that way, but most didn’t care or push back. The debt instrument took hold and is here to stay. At least for a while. Now, I see very few seed deals that are not structured as convertible notes. Those that aren’t are generally led by institutional investors who often require an equity instrument to do an investment.
There is just so much fighting of the establishment that one can do and, as such, I’ve conceded that if I am to remain an investor, I have to adapt. That doesn’t mean I have to roll over, though. As such, I’ve created a set of criteria for myself when it comes to reviewing the convertible notes of potential investments. There are certain parts of a note that I like to see and some I don’t like. More importantly, there are parts that cause me to negotiate strongly and, ultimately, walk away if they are not added, removed or modified. I’ve tried to outline each of these below.
For angel investors reading this, I’d appreciate if you’d add your thoughts on what I’ve missed and where I’ve overstepped. For founders of startups, I hope you’ll look at these items as guidelines when you’re putting together your note. My intent is not to create an investor-biased instrument, but to fill the holes left by current convertible notes; to shed light on why some terms don’t work well for angel investors and to suggest how to change them so they work better. I don’t believe that any of the guidance I have or changes I recommend should be detrimental to the founders of the company. Speak up if you disagree.
Discount and Interest
Seed investors take a significantly higher level of risk than later stage investors. Let’s face it, the vast majority of startups fail and most of those fail early. In an equity investment round, the risk level taken by angel investors was built into the valuation of the company. Since this is not part of the note, then the investment risk has to be recognized through another avenue. This is the importance of the discount. A discount rate of 20% is pretty standard, but is it appropriate that the seed investor pays 80% what the next round’s investors pay when there might be a year between investment rounds? Probably not. So, the discount rate needs to be judged alongside the cap on the note. The combination of the two should recognize the risk taken by the early investor. That said, a discount rate of less than 20% is pretty big red flag for me.
On interest rates, I usually see 5-6%. I don’t focus on these too much because that’s not why I’m making the investment. Still, if it’s much outside this range, I’ll want to know why.
First and foremost, there must be a cap on the conversion. Again, why should an early investor take the risk of an early investment only to have someone else decide on the actual value of that investment somewhere down the road?
There is no absolute on the cap, of course. It depends on the company’s stage and what’s happening in the overall market. The cap should reflect the projected value of the company at the approximate time of the next financing, discounted to take the risk of the market and execution into account. Good luck with that. Really, it’s the number that in combination with the discount will get investors to fill your round. Hopefully, you can begin to see why negotiating this is no more efficient than negotiating an actual current valuation
Events Prior to Conversion
One of the biggest problems with convertible notes from the investor’s viewpoint is plugging all the potential scenarios that can take place between investing via a note and the conversion of the debt to equity. For example, if the company is sold prior to conversion, does the investor simply get their money back without recognition of the value of their investment? This situation is covered naturally in an equity round, but not in most convertible note documents.
As such, the note must specify that any change of control of the company (aka sale of the company or liquidation event) before conversion has taken place triggers a capped and discounted (as per the terms of the note) conversion into common shares in the company immediately prior to the execution of the corporate transaction. What this means is that the investor’s debt will be exchanged for equity according to the terms of the note, making the transaction a proxy for an equity investment round.
If the transaction is for less than the conversion cap, then the investor should have the right to receive a multiple of their investment. This can be on a sliding scale based on the time between their investment and the transaction. I have seen a fixed multiple of 2X applied to cover this gap.
Closing of the Note
The convertible note should not remain open for a long period of time. Sometimes, company founders will create a note and continue to take money under the terms of the note for an extended period of time. Since the terms of the note represent the level of risk taken and time, generally, reduces risk, it’s not fair for later investors to get the same terms as substantially earlier ones. Therefore, notes should be closed within 90 days of their opening to recognize the risk being taken by the early investors. If a note remains open for a long time, many angel investors will do their best to be the last money in which can make it difficult for the founders to raise the cash they need.
Pari Passu with Future Notes
It used to be that the early seed round was done with a note and the subsequent rounds were equity deals. These days, angels are seeing multiple rounds of convertible notes. When this happens, it’s important that the original note holder ensures that any more favorable terms in the future notes apply to the original note as well. Why should the people who took more risk by investing early receive fewer benefits?
To be clear, this does not apply to retrofitting the terms of an equity investment to original note holders. On conversion, the note holder will become equity participants via the conversion and receive the same terms, with some exceptions mentioned below, as the participants in the equity round.
The Major Investor Clause(s)
Many note documents include a class of investor referred to as the Major Investor. The Major Investor is usually granted rights that include guaranteed information, inspection rights and the right to invest in the next round of funding at their pro rata level at minimum. Effectively, the Major Investor clause(s) give the company and/or investors the right to preclude non-Major Investors from getting information or investing in the next round. As an angel investor, this is a huge negative. If things are going well, I may not have the ability to invest further. I’m not sure how that benefits anyone. No such clause should exist. If it does, I’ll ask for a side letter including me as a Major Investor.
I cover this thoroughly here.
A permutation of this that is regularly found in notes covers what is often known as the Requisite Holders. Sometimes, I’ll find the description of the rights of a Requisite Holder at the end of the note buried in the boilerplate stuff that no one ever reads. Here’s an example:
Any provision of the Notes may be amended or waived by the written consent of the Company and the Requisite Holders.
Where the group of Requisite Holders may be defined as the largest investors in the round. So, basically, certain investors may change the terms of the note at any time even to the extent of making the terms of the note different for each investor. The especially bad part of this, as I mentioned, is that this clause is often found at the end of the note, where no one is really paying attention. Why would I agree to this? Does this serve the company?
Whew! That was long. If you’ve read this far, I hope it was helpful. I’d like to believe that this is a treatise that will reverse the tide of the use and overuse of convertible notes. Of course, I know it won’t. So, at the very least, if this helps a few angel investors be more diligent about convertible note documents and convinces more to push for equity investments instead of debt, then I’ve done my job.
I’m a fairly active angel investor. I’ve certainly done enough investing to have seen a broad range of investment paperwork and the preferences of most of the top lawyers used by startups in the country. Generally, the docs are mostly boilerplate. Legally mandated cover-your-ass type stuff. So, rather than wade through reams of paper for each investment, I have key terms that I look for and pretty much ignore the rest. Almost everyone’s intentions are good and all parties involved want all to benefit from the success of the company or to participate equally in its failure. Rarely does someone want an unfair advantage in a deal. Well, mostly.
In about a third of the deals I do these days, I run across the dreaded Major Investor clause. It goes something like this:
“Major Investor” means any Investor that, individually or together with such Investor’s Affiliates, holds at least X shares of Registrable Securities (as adjusted for any stock split, stock dividend, combination, or other recapitalization or reclassification effected after the date hereof).
Big deal, you say, that’s just a definition. True, but why define a Major Investor at all? Because later in the document (usually in the Investor’s Rights Agreement), certain rights and privileges are reserved solely for the Major Investors.
Who are these people? Generally speaking, they are a lead investor in the deal. A single large investor, a family office, a VC, an angel fund or some other investment group. Oh, by the way. Did I mention that the X in the definition above is generally set higher than the level that individual investors are coming into the deal at, leaving the lead investor as the only Major Investor. But, I bet you guessed that already.
What are the rights they reserve for themselves?
- Information rights
- Inspection Rights
- Preemptive Rights (also known as Right of First Offer or Right of First Refusal)
Information rights grant investors the right to receive, at some pre-determined interval, information about the company’s status and finances. Limiting information rights to just the lead investor is just silly. The idea is that it saves the management team time, not having to report to too many people. Why this takes more time than reporting to one party, I have no idea. In practice, it’s just another cc: on the distribution list of the status email.
Some VCs voice a concern that granting such rights to all investors will create a dialog of follow-on questions and comments that will consume too much of management’s time. In my experience, there is almost never much of this going on unless it’s encouraged by the CEO. Often, the angel investors, having mostly run companies themselves, can offer up their situational wisdom when they know what’s going on. This advice is frequently more valuable than the money that was invested.
Inspection rights, which grant investors access to the company’s books, facility and personnel from time to time, is more difficult. I actual agree with limiting inspection rights. This can create a huge time sink for the company and get out of hand when a deal has many investors.
So, Preemptive rights are a protective provision that give early investors the right (not the requirement) to invest more in the company in order to maintain their level of ownership, avoiding dilution. All early investors should have this right, which should be explicitly granted in the investment documents. By keeping this right for themselves, Major Investors are virtually guaranteeing that the angel investors in the round will be screwed in the subsequent rounds of investment if things are going well.
You might ask if this happens in practice. I can assure you that it does. I have fallen victim to this more than once, although not in a while. I know many angel investors who have gotten caught in this trap because they didn’t understand or didn’t take a hard look at the paperwork before making an investment. Recently, I was in a situation where I had to fight to maintain my ownership in a follow-on round even though I had Preemptive rights. The VC investing in the new round demanded of management that all the angels who had previously invested give up their protective right. Several of the angels refused and the deal went through anyway. If there was no prescribed right, you can imagine how it would have gone down.
If you’ve gotten this far, you probably understand my point of view concerning the Major Investor clause. It’s changed my process of discussing an investment with a startup and reviewing the documentation for the investment. My first move is to search for the term “Major Investor.” If it’s found, I check on the rights granted (implicitly precluding the angels in the deal). I don’t really care about Inspection rights and I think that limiting Information rights is stupid, but that won’t prevent me from doing the deal. Preemptive rights is a much bigger deal. I’m simply not interested in making an investment in a company that doesn’t offer me a Preemptive right in return for the risk I’m taking as an early investor. I don’t always invest my pro rata share in subsequent rounds, but I’m not willing to give up my right to do it.
I am fortunate to have a 2002 BMW E46 M3 convertible. It’s a great car that I love to drive. Unfortunately, it has auto electronics circa 2001 (when I actually purchased the car). About a year into owning it, I decided that the trunk-mounted CD player just wasn’t going to cut it. I had a huge number of ripped CDs and even though downloadable MP3s weren’t quite readily available yet, I knew that they would be broadly available soon. In searching for a way to play MP3 files through the standard radio interface of the car, I found the Holy Grail. Well, it seemed that way at the time. A removable disk-based MP3 player that used the existing BMW wiring to connect to the standard head unit in the dash. All praise the PhatNoise PhatBox.
The idea was great, a device with a removable disk drive (20GB!) that used the same cabling as the target vehicle. In my case, a 3-Series BMW. The drive could be removed from its semi-permanent home in the trunk of my car, inserted into a docking port connected to my computer and synced with my MP3 database of tunes. What could possibly go wrong, right? A spinning disk in an enclosed space without any climate control and subject to the shaking, rattling and rolling of a tightly sprung sports sedan. Perfect conditions for a fast-spinning disk drive . . . Not.
Well, this device has been a rock-solid stud. Almost never a skip or a missed beat, even on washboard New England roads in 90 degree heat and equal humidity. The PhatBox never failed to play all my favorites. On call, all the time. Along the way, I lost the docking station and stopped updating the music on the PhatBox. No matter really, since I mostly listen to classic rock and there’s not a lot of that coming out these days by definition.
The PhatBox is a seriously great product. Not only did it put up with being in the trunk of a car for the last dozen years, for its day, it was an exhibitionist of great engineering. PhatNoise dealt with the limited interface on the BMW Nav/Infotainment unit by synthesizing voice directives so that they could use the limited number of buttons available to multiplex several functions. When a button is pressed once, the PhatBox announces, “the current playlist is XYZ.” When pressed again, the next playlist is announced. Easy smeazy. The synchronization app worked flawlessly – much better than Apple’s abomination, iTunes. No muss, no fuss and it didn’t even cost a lot. Yeah, sure, I had to fabricate a mounting bracket for it so that it fit where the old CD changer was mounted, but that was just a few hours of work. Unfortunately, it appears that PhatNoise couldn’t keep up. They appear to still be in business, but no longer sell the PhatBox. The solid state, iPod dominant ship came ashore and sailed and PhatNoise didn’t get on board.
So, today, the PhatBox got lovingly removed from the M3 (these photos are of my PhatBox after removal from the car). It’s brought me loads of joy over the years, but my inability to update it – a man can’t live on classic rock alone, after all – has forced me to move on. I replaced it with an Audiovox Mediabridge that uses an external iPod or USB memory stick loaded with MP3s. It also has a Bluetooth connection and can wirelessly connect to my phone. It’s not like landing on Mars or anything, but it is an advance in technology.
PhatNoise created a terrific product that has brought me loads of enjoyment over many years. It’s too bad that they couldn’t make a go of it and bring all their great engineering prowess into the present. I have incredible respect for the people who built the product. It’s been a total blast to use for all these years.
OK, I admit it. I’m a data junkie. I just totally believe that you can’t improve what you can’t measure. So, I want to measure everything. Although, it’s even better when someone or something does the measuring for me. With this in mind, I recently decided to figure out what was the best way to measure my physical activity – movement, calories, weight lifted, stairs climbed, etc. Unfortunately, some of this data remains hard to come by. While machines in the gym – treadmill, elliptical, stationary cycle, etc. all generate some types of data. This data isn’t normalized across machines and, generally, cannot be exported for tracking.
The answer seemed to lie in the new crop of activity monitors available. The small devices worn on the wrist or kept in the pocket to track the steps one has taken, the calories burned, elevation climbed and so forth. Unfortunately, the current crop of activity monitors don’t really even try to cover some of the data I was looking for. Even worse, I had heard that many of the facets of activity they claim to cover are not all that accurate. Since my interest in gathering activity data was bordering on a need rather than desire, I had to find out what my options were. So I tried several trackers out.
My pseudo-scientific test included most of today’s popular devices plus one monitor that runs on my Android phone (there are other Android apps and, of course, iOS apps as well, especially for Apple phones with the M7 processor).
- Withings Pulse
- Fitbit Flex (thanks Brad Feld)
- Garmin Vivofit
- Jawbone UP (thanks Shawn Broderick)
- Moves App
There are several more, of course, but my arms are short. I wore these devices daily for about a month. It was still cold outside so I could hide this embarrassing electronic armband with the sleeves of a shirt. There are lots of features of these devices that I don’t cover here. If you’re going to make a purchase, you should hit the companies’ web sites for complete information.
I need to note that I didn’t test all types of activities that these monitors track. Since these devices are for the arm or pocket, cycling isn’t accurately tracked (nor is it claimed to be) and I am not a runner, so I didn’t even test for running. I also didn’t test the sleep functions of any of the trackers. I move around a reasonable amount each day, I frequently spend time on an elliptical trainer and in the gym lifting weights or doing body-weight exercises. I am a reasonably fit and active person. The question is, would the data reflect this.
Let me get right to the bottom line: These devices measure some activities moderately well and others either poorly or not at all. They all greatly depend on a certain type of body movement that their accelerometers (motion detectors) can pick up. A person who has a hard-pounding walking style will register more activity than one who floats over pavement, for example. If your primary activities are non-aerobic (weight lifting), these devices are useless. And even if they are aerobic, but only involve smooth movements (e.g. cycling), very little data is acquired. They are, basically, walking and running monitors. Additionally, the data gathered is best used as a comparison of the user’s activity over time because the absolute accuracy of these devices is questionable.
Here are a couple of examples that bear this out . . .
The reports below are generated by each of the devices. The first three – by the Garmin, Withings (side-by-side) and Fitbit (below), respectively – are clips from their web pages. Unfortunately, the next two – from Moves and Jawbone – don’t provide a graphical web interface. The data is only available on a computer by downloading it into a spreadsheet via the web. All the devices have phone apps that display the data graphically. While it’s nice to be able to access the data on a phone, I much prefer being able to see and manipulate the data with a computer on the web. Personal preference.
This is the data from May 10 – a randomly chosen day (I did the test on the elliptical 14 times during the month). All devices were worn throughout the day. You can see that the data on number of steps varies wildly.
All reported correctly that the primary movement started at about 7:30 pm and lasted for about an hour. During that time I was on an elliptical machine which can very accurately track the number of steps taken. The elliptical reported, roughly, 6,500 steps were taken, making the reported values of less than 5,000 for the whole day a bit suspect.
The distance covered also varies a lot between devices. It ranges from 2.09 mi (Fitbit) to 4.19 miles (Withings). Regardless of what distance I actually traveled during the day, the 2x difference in range makes me question all of the data. FWIW, the elliptical claims I ran/climbed more than 6 miles during that session.
On another day, I did a similar test on the elliptical with all devices in my pocket instead of on my wrist. The results were different – all values were higher, but the variance was just as high.
The data from April 21 is below in the same order as previously reported. Unfortunately, there is no Jawbone data for this day. This was a moderately active day with a concentrated weight lifting session from 2:30-3:30 pm. Note that none of the apps register much activity during this time. For most of the day, I was just moving around, doing whatever I needed and wanted to do without actually “exercising.” For this, most of the trackers were more aligned, however, there is still almost a 2x difference between the lowest (Moves) and highest (Garmin) in step count.
Clearly, the lack of activity reported during a very active weight lifting session shows that these trackers are not a reliable way of tracking this type of activity data.
Some thoughts about each of the trackers . . .
- Device fits in pocket nicely or on the wrist with a watch-like band
- Great display of all data, scrollable to see results from other days
- Touch sensitive screen for scrolling through data
- Micro-USB port on device for charging (this is a pro because as an Android user, I always have a micro-USB charger with me)
- Hard to read display in sunlight
- Not water resistant
- Replaceable battery lasts for a year (others need recharging after 10-14 days)
- Bright display easily readable in sunlight
- Red reminder indicator to get your ass off the couch
- Water resistant
- Data is combined with that from other Garmin devices to give a bigger activity picture
- Bluetooth syncing failed frequently
- Web site is difficult to negotiate
- Small, easily moves from wrist band to pocket
- Water resistant
- No data display – just some LEDs showing progress towards the day’s goals
- I found the wrist band hard to put on even after a couple of month’s usage
- Requires USB dongle for recharging
- Runs on phone so has optional access to GPS data – knows how fast your moving and where you are
- Runs on phone so it’s almost always with you
- Runs on phone so it consumes battery power
- No graphical data available on the web
- Looks the most like jewelry
- Water resistant
- No data display – just a colored light to tell you when you’ve achieved your goal
- No graphical data on the web
- No wireless communication – must connect to computer to download data (newer version of hardware apparently has Bluetooth)
- Requires USB dongle for recharging
Where do I go from here . . .
None of these devices are perfect or, for that matter, even very good in an absolute sense. As I said earlier, they do a decent job indicating your relative activity from day to day and in that way, they can disclose and track some valuable metrics. Many of these devices have other features that may increase their value to the user as well. The Withings Pulse can also track your pulse and blood oxygen levels and the Garmin Vivofit always shows the current time, for example.
For me, I think I’m going to move to a combination of devices. Perhaps Moves on my phone because it’s so transparent for daily activity (I’m carrying it anyway) and either the Garmin Vivofit or Withings Pulse for when I’m purposefully exercising. I’m then going to use the HumanAPI to combine the data so that I can track my overall activity in one place. Or, perhaps I’ll get some help with my OCD-ish need for collecting data and drop the whole thing altogether.
This is my third Thinkpad – first from IBM and now Lenovo. They have been my laptop of choice for as long as I can remember. An X40, then an X60s and now this new baby. Not as stylish as those unibody Macs that almost everyone I know uses these days, but I’ll take function over form any day (well, mostly – although my kids strongly disagree, I’m not entirely without style). These computers have been rock solid over the years and I’ve been able to continuously extend their lives, upgrading batteries, disks, memory and versions of Windows – eeking out more from these machines than IBM and Lenovo probably ever intended. They have been no-muss, no-fuss workhorses and I fully expect the same from the X220.
The configuration I purchased isn’t even all decked out. I selected the options that best met my needs – Sandy Bridge i5, 2.5GHz, 6GB of memory, 128GB SSD, 1366X768 IPS 16X9 12.5” display and Windows 7 64-Bit (oh yeah, baby). While that’s still a formidable laptop setup, faster processors, more memory and bigger disks are available to drive this thing faster and further.
The system boots fast and resumes from standby instantly. The screen is really sharp and the computer executes everything quickly. Best of all, battery life is completely outstanding. I can pound on this things for 5-6 hours without refueling. If I’m just watching videos, it’s a couple hours more than that – excellent for long plane rides. I’ve stopped carrying my iPad. At 1.0” thick (there is another 0.25” bump where the battery is) and weighing in at about 3 pounds, it’s light and goes almost anywhere my iPad went and I like the keyboard way better.
As with most things, not all is perfect. The machine comes with IBM/Lenovo’s classic TrackPoint device, which I’ve always loved. It also comes with a touchpad. You can set the machine to recognize one or the other or both. Problem is, the touchpad sorta sucks. It doesn’t track consistently and trying to use it alongside the TrackPoint requires manual dexterity that genetics hasn’t quite yet refined. So, I have the touch pad turned off. The other problem is with the display. While it’s bright and sharp and colors are superbly reproduced, 768 pixels filling the, roughly, 6.25” screen height just doesn’t cut it. As much as media wants to go widescreen, productivity apps still long for good ol’ 4:3. Or, at least a physically taller display so that what’s displayed is easier to read. There’s just not enough vertical information displayed when trying to get real work done or even just browsing the web.
Do these problems detract from the experience? Perhaps. Everyone needs to decide for themselves. For me, the screen height thing keeps this from being a perfect, do-everything computing device, but it’s just not enough of an issue to spoil all the advantages that it offers. I suggest you take a look at one before buying to judge for yourself, though. It may be a more substantial issue for you.
The battery life on its own makes this computer terrific. Add to that the speed, great keyboard, bright display, Windows 7 and upgradeability and I think this will be my laptop for many years to come. Even if I have to do a lot of vertical scrolling.
After seven years of riding in the Pan-Mass Challenge, a 2-day charity bike ride across Massachusetts, I’m going to have to bail out of this year’s event. The ride is a big deal for me each year because it supports a truly meaningful cause – cancer research – and the supporters of it, most of whom have been touched by cancer, really work to make it a rewarding experience. It’s a tough ride, but a total blast. I’m skipping this year because my knee, which I had surgery on in March, has not healed LIKE I WAS PROMISED! The surgeon said I’d be back in four months and now is saying it’ll be at least six. The physical therapist isn’t even that optimistic. As it turns out, the fine print in the surgery contract doesn’t say anything about commitments by a medical professional being legally binding (yes, I’m kidding).
While I’m not going to be able to do any actually pedaling in the event, I can still do some peddling (get it? pedaling vs. peddling? funny, right?). I’m still going to try and raise some money for The Jimmy Fund. To do that, I’m going to be a “virtual rider” for the PMC. It’s just what it sounds like, I’m afraid. I pretend to ride so that I can pimp the cause.
If you have the desire and ability to donate, I’d appreciate your support of the efforts at Dana Farber. I have raised $35,920 over the last 7 years and already have $4,840 committed for this year. I suppose it’s a bit lame to be seeking donations when I’ll be sitting on my butt during the 2-day ride, but if by making it a bit easier to donate I can convince a few more people to throw in with the cause, it’s worth it.
Thanks in advance!