Activity/Fitness Monitor Showdown

OK, I admit it. I’m a data junkie. I just totally believe that you can’t improve what you can’t measure. So, I want to measure everything. Although, it’s even better when someone or something does the measuring for me. With this in mind, I recently decided to figure out what was the best way to measure my physical activity – movement, calories, weight lifted, stairs climbed, etc. Unfortunately, some of this data remains hard to come by. While machines in the gym – treadmill, elliptical, stationary cycle, etc. all generate some types of data. This data isn’t normalized across machines and, generally, cannot be exported for tracking.

The answer seemed to lie in the new crop of activity monitors available. The small devices worn on the wrist or kept in the pocket to track the steps one has taken, the calories burned, elevation climbed and so forth. Unfortunately, the current crop of activity monitors don’t really even try to cover some of the data I was looking for. Even worse, I had heard that many of the facets of activity they claim to cover are not all that accurate. Since my interest in gathering activity data was bordering on a need rather than desire, I had to find out what my options were. So I tried several trackers out.

My pseudo-scientific test included most of today’s popular devices plus one monitor that runs on my Android phone (there are other Android apps and, of course, iOS apps as well, especially for Apple phones with the M7 processor).

• Withings Pulse
• Fitbit Flex (thanks Brad Feld)
• Garmin Vivofit
• Jawbone UP (thanks Shawn Broderick)
• Moves App

There are several more, of course, but my arms are short. I wore these devices daily for about a month. It was still cold outside so I could hide this embarrassing electronic armband with the sleeves of a shirt. There are lots of features of these devices that I don’t cover here. If you’re going to make a purchase, you should hit the companies’ web sites for complete information.

I need to note that I didn’t test all types of activities that these monitors track. Since these devices are for the arm or pocket, cycling isn’t accurately tracked (nor is it claimed to be) and I am not a runner, so I didn’t even test for running. I also didn’t test the sleep functions of any of the trackers. I move around a reasonable amount each day, I frequently spend time on an elliptical trainer and in the gym lifting weights or doing body-weight exercises. I am a reasonably fit and active person. The question is, would the data reflect this.

Let me get right to the bottom line: These devices measure some activities moderately well and others either poorly or not at all. They all greatly depend on a certain type of body movement that their accelerometers (motion detectors) can pick up. A person who has a hard-pounding walking style will register more activity than one who floats over pavement, for example. If your primary activities are non-aerobic (weight lifting), these devices are useless. And even if they are aerobic, but only involve smooth movements (e.g. cycling), very little data is acquired. They are, basically, walking and running monitors. Additionally, the data gathered is best used as a comparison of the user’s activity over time because the absolute accuracy of these devices is questionable.

Here are a couple of examples that bear this out . . .

The reports below are generated by each of the devices. The first three – by the Garmin, Withings (side-by-side) and Fitbit (below), respectively – are clips from their web pages. Unfortunately, the next two – from Moves and Jawbone – don’t provide a graphical web interface. The data is only available on a computer by downloading it into a spreadsheet via the web. All the devices have phone apps that display the data graphically. While it’s nice to be able to access the data on a phone, I much prefer being able to see and manipulate the data with a computer on the web. Personal preference.

This is the data from May 10 – a randomly chosen day (I did the test on the elliptical 14 times during the month). All devices were worn throughout the day. You can see that the data on number of steps varies wildly.

All reported correctly that the primary movement started at about 7:30 pm and lasted for about an hour. During that time I was on an elliptical machine which can very accurately track the number of steps taken. The elliptical reported, roughly, 6,500 steps were taken, making the reported values of less than 5,000 for the whole day a bit suspect.

The distance covered also varies a lot between devices. It ranges from 2.09 mi (Fitbit) to 4.19 miles (Withings). Regardless of what distance I actually traveled during the day, the 2x difference in range makes me question all of the data. FWIW, the elliptical claims I ran/climbed more than 6 miles during that session.

On another day, I did a similar test on the elliptical with all devices in my pocket instead of on my wrist. The results were different – all values were higher, but the variance was just as high.

The data from April 21 is below in the same order as previously reported. Unfortunately, there is no Jawbone data for this day. This was a moderately active day with a concentrated weight lifting session from 2:30-3:30 pm. Note that none of the apps register much activity during this time. For most of the day, I was just moving around, doing whatever I needed and wanted to do without actually “exercising.” For this, most of the trackers were more aligned, however, there is still almost a 2x difference between the lowest (Moves) and highest (Garmin) in step count.

Clearly, the lack of activity reported during a very active weight lifting session shows that these trackers are not a reliable way of tracking this type of activity data.

Some thoughts about each of the trackers . . .

Withings Pulse:

Pros –

• Device fits in pocket nicely or on the wrist with a watch-like band
• Great display of all data, scrollable to see results from other days
• Touch sensitive screen for scrolling through data
• Micro-USB port on device for charging (this is a pro because as an Android user, I always have a micro-USB charger with me)

Cons –

• Hard to read display in sunlight
• Not water resistant

Garmin Vivofit

Pros –

• Replaceable battery lasts for a year (others need recharging after 10-14 days)
• Bright display easily readable in sunlight
• Red reminder indicator to get your ass off the couch
• Water resistant
• Data is combined with that from other Garmin devices to give a bigger activity picture

Cons –

• Bluetooth syncing failed frequently
• Web site is difficult to negotiate

Fitbit Flex

Pros –

• Small, easily moves from wrist band to pocket
• Water resistant

Cons –

• No data display – just some LEDs showing progress towards the day’s goals
• I found the wrist band hard to put on even after a couple of month’s usage
• Requires USB dongle for recharging

Moves

Pros –

• Runs on phone so has optional access to GPS data – knows how fast your moving and where you are
• Runs on phone so it’s almost always with you

Cons –

• Runs on phone so it consumes battery power
• No graphical data available on the web

Jawbone UP

Pros –

• Looks the most like jewelry
• Water resistant

Cons –

• No data display – just a colored light to tell you when you’ve achieved your goal
• No graphical data on the web
• No wireless communication – must connect to computer to download data (newer version of hardware apparently has Bluetooth)
• Requires USB dongle for recharging

Where do I go from here . . .

None of these devices are perfect or, for that matter, even very good in an absolute sense. As I said earlier, they do a decent job indicating your relative activity from day to day and in that way, they can disclose and track some valuable metrics. Many of these devices have other features that may increase their value to the user as well. The Withings Pulse can also track your pulse and blood oxygen levels and the Garmin Vivofit always shows the current time, for example.

For me, I think I’m going to move to a combination of devices. Perhaps Moves on my phone because it’s so transparent for daily activity (I’m carrying it anyway) and either the Garmin Vivofit or Withings Pulse for when I’m purposefully exercising. I’m then going to use the HumanAPI to combine the data so that I can track my overall activity in one place. Or, perhaps I’ll get some help with my OCD-ish need for collecting data and drop the whole thing altogether.