You should own your data.

Let’s face it, we use a ton of services daily. You use Google to search for things, YouTube to watch videos, Google’s SSO, Google Drive, Google Maps and so on. It’s impossible to escape Alphabet. Even if the site you visit is not owned by Google, it probably uses Google Ads and AWS for something and Amazon isn’t much better is it?

The harsh reality is, the only way to avoid giving up your data to questionable companies and individuals is to self-host everything. This is not feasible for $99\%$ of the population and if you are a part of that $1\%$, you will quickly have to accept that you will have to make sacrifices in order to be fully in control of your data. The sacrifices range from worse ease of use, worse UX/UI to features which simply cannot be replicated in a self-hosted environment.

Everyone has to draw a line somewhere on their threat model. Sometimes, the effort to set something up on a machine, have that machine up and running at all times and the cost of having that machine in the first place completely outweigh the benefits from self-hosting.

EXAMPLE

For example, I don’t really care that, say, Spotify knows what I listen to and when I listen to it, but I do care about being able to listen to music at all times and be able to easily discover new music. Sure, Spotify can (and does!) arbitrarily remove songs off their platform, but for me a simple backup of my playlists every now and then does the trick.

What I do not like, however, is having my own data gatekept from me, with very few of it available for free, some of it available after a fee and most of it completely lost somewhere on someone’s computer. And I especially don’t like it when it’s a platform which has very personal information of mine in it.

This is exactly what happened to me with a fitness platform and I wrote this article to share my rather horrible experience with them.

Disclaimer

I will not reveal the names of the people involved, the name of the company involved or show images of the conversations due to legal concerns.

A lot of the numbers quoted in this article are ballpark estimates.

This is a lengthy read

If you want to skip to my personal recommendations, click here.

Where did it all start

The company has had a big grip on the fitness app industry, and I get it. It’s free and it has a lot of user-submitted data.

Well....

The user-submitted data suffers from a big quantity over quality issue (at least in my region). While you are likely to find the brand and product you are looking for, I can guarantee that the data is incomplete or straight-up wrong in about $40\%$ of the times. You cannot have your cake and eat it too!

So naturally when I began caring about this type of stuff, I naturally gravitated to it.

And initially, all was good. Everything worked the way I expected it to and I had access to just the right amount of information to do the job.

…however at some point, given you use a service for long enough, you will almost always run into a situation where you think to yourself…

QUOTE

Damn. I should be able to do this!

For some that may be the removal of features. For others, it’s the inability to access historical data easily. And for another group it might be that they do not like seeing ads all. the. time.

Trust me.

I get it. It’s a company for a reason. I understand they have to make money somehow. However given they are making money off of me, I feel like I have the right to at least complain about it.

After a couple months of thinking "hmm, I could easily do this." whenever I had issues with the platform, I decided to give it a shot and migrate to a self-hosted solution.

Step 1. The data

Naturally, I didn’t want to lose any of my data, it is mine after all. I wanted the transition to whatever platform I choose to be as seamless as possible.

…

what platform am I choosing anyways?

Step 0. Choosing the platform

The list of requirements I had were the following:

• Self-hostable. Willing to accept a non self-hosted solution as long as it is open-source.
• Good with data. I want complete access to my data and I want all of it (goes hand in hand with the first requirement)
• Can import data from the old platform.

After looking at all my options, I realized that I like none of them.

lovely.

Naturally, this meant one reasonable thing. I had to write one myself.

Having chosen a platform, I began by trying to extract my data using the app. Clicking on the Download your data button lead me to a menu about upgrading to the Premium plan. No good.

I knew that I could do it for free by requesting the data from the website, so I submitted a request and waited…

…and waited…

…and waited…

and after $2$-ish weeks, it was here! A magnificent 50 or so thousand line .xlsx file.

$50,000$

$50,000$ lines is a handy number which, as one slightly copyrighted mouse says, will come in handy for us later.

Problem #1: Missing data

The first issue I encountered was the fact that the data was fairly incomplete. It contained:

• Name

• Username

• Email Address

• Country

• IP Address, at last login

• Weight

• Height

• Nutrition Summary

$\dots$

Despite what looks like a fairly exhaustive list of what you’d expect, it did not include:

• The logged food’s name

• The logged food’s values per 100g (or equivalent)

Technically...

Technically speaking, this is not my data so they are in not obliged to give it to me. I don’t blame the company for this.

Dang. I guess we will have to find a workaround. I looked through the file and its $\approx 50,000$ thousand lines hoping to find an ID-to-more-data map. The first $\approx 1,000$ files were account metadata, so my name, email and so on.

The remaining lines in the range of $40,000\dots 50,000$ were all the summarized food information mentioned above.

Did you catch that? If you didn’t, feel free to pause the article and see if you can find it. If you found it, congratulations, you are more competent than whoever was dealt with giving me my information.

What’s up with the lines between $1,000\dots 40,000$?

Problem #2: It gets much, much, much worse

If you’ve used Excel for slightly longer than 5 minutes, you would probably discover the Hide row/column feature. (here it is in all it’s glory)

Do you see where I’m going with this?

You

Oh, so they hid the data? Just hover the cells, click unhide and voilà, we have our data.

This is probably what you are thinking to yourself as you read this.

If this is what actually happened, I would not be writing this article.

If I gave you $3$ chances to guess what was in those rows, you would probably not get it, or if you do, you won’t believe this is something that could actually happen.

It was other users’ data.

Yes. Approximately $5$ other users’ data. Entirely random users.

Everything. Stored using military grade security.

Absolutely disgusting.

Hold on. What now?

I took a step back, took a deep breath and started thinking about what my next step should be.

I considered the thing you are thinking about, but it was infeasible for me.

I ended up contacting the company, telling them about the incident. I deleted the file pre-emptively, as I was unsure about the legal consequences of being in possession of other people’s personal identifying information. As expected, the company confirmed the accident that had occurred, instructed me to delete the file and sent me a new, this time legal file.

Proposed solution?

The issue was escalated to the company’s community manager who offered me a year of their premium plan for free.

What a nice and beautiful gesture! Would’ve been a very good thing to do, had the issue been a small-medium sized bug reported, not a crime.

Don't get me wrong

I don’t seek financial compensation. I don’t seek any compensation. The reason why I think the premium offer was hilarious is not because I think I deserve more, it is because of how well it describes the whole problem in the first place. Is your personal information worth $2\frac{1}{2}$ months worth of subscribtion money?

I didn’t accept their offer. Not because of some feeling of pride (well maybe some of it), but because I don’t want to do any business with a company who values my name, username, email, IP address and body information at $2\frac{1}{2}$ months of their fancy, paywalled CRUD application.

Step 2. The rest of the owl

I found a way to access all of the data I needed. Turns out, it’s all stored locally in a SQLite database on your mobile device. After I got the database, I processed the data and ported it over to my server. It was relatively easy.

Slight tangent

The SQLite database is a hot mess. It’s exactly what you expect of a big project which has had to deal many features being added with little consideration for the existing infrastructure. Everything just kinda works and is one gust of wind away from breaking.

Why not name and shame?

That’s a question I will probably get a lot. The answer for me is simple:

1. I don’t see much value in doing so.

2. I could get dragged into lawsuits, etc.

3. Screenshots of email conversations can be faked just as easily as saying they exist.

I’m sure you understand the last two, but what about the first one?

This is an issue that is completely platform-agnostic. It can happen to you, regardless of if you are using the platform I was talking about. Hell, it probably does happen to you, if we are being honest, there’s probably some more egregious stuff going on behind the scenes at some companies who’s products you use daily.

The way to fix this issue is not to avoid a certain platform until they’ve given you a

pinky promise we will not do this again!! :DDD *wink wink* bythewayweapologisefornothingasitcangetusinlegaltrouble we are sorry you felt that way!

The way to fix this issue is to stop relying on a third party for absolutely everything.

Maybe some of you

I don’t trust you!

Even if you don’t trust me, my recommendations are completely independent of anything up until this article. I believe you shouldn’t need to run into an incident like that, or wait until someone else does, for you to care about your privacy, so even if you think this didn’t happen, give the recommendations a read and apply that lack of trust to some companies.

Recommendations

everything here is an opinion

1. Most people’s threat models are very generous.

   I’m not saying you should go full trust no one. compile linux on your own. every day. live in a shed in the woods. make your own glue. obviously. I haven’t. This blog is (currently) hosted on a tech giant’s free tier of hosting. My threat model considers that the contents of this blog are not anything special and, after all, they are publicly accessible anyways. I could always rent a machine over at Hetzner or repurpose an old machine for some Quartz hosting, but I don’t see much value in that.

   However I do think that people give away their data for way too cheap. If you think of every interaction with an online platform as a transaction, one where you trade away a bit of information, a bit of your soul for a bit of knowledge or entertainment, you may end up recognizing that some things are not worth it.

   Companies literally fight over your data. Companies pay millions acquiring other companies because of your data. Are you willing to sell it for that cheap?

2. Don’t start with the most convenient option first.

   Companies rely on their convenience. After all, who wouldn’t turn down some free information processing, bandwidth, CPU cycles and beautiful graphics, when the alternative would be something paid, which you have to go out of your way to manage and also probably has less features. As stupid as it sounds, I avoid using convenient services and platforms in favor of less convenient, but privacy focused alternatives because I’m afraid I will become reliant on their product.

   Convenience is a trojan horse. Treat it as one.

3. Whenever possible, always go for an alternative which is offline-first.

   In most situations, if a platform can be used offline, it’s a good indicator that you have some control over your data.

4. Do you really need everything?

   How many times have you used every feature you pay for? If you use Spotify, how many times have you used their AI DJ? What about the music videos? Have you ever clicked on the karaoke mode button? You are probably paying for things you didn’t even know existed.

   What you do is you pay for the convenience of being able to listen to music without interruptions. You pay for the convenience to remove an arbitrary inconvenience. Have you considered the alternatives? Is there a way to get what you pay for, except cheaper?

5. Explore alternatives.

   You want to move on from Spotify? Check out Navidrome.

   You want to move on from Netflix? Check out Jellyfin.

   You want to move on from Android/iOS? Check out GrapheneOS

   You want to move on from <thing>? Check out PrivacyGuides. Hell, PrivacyGuides is so good that I’m just going to-

IMPORTANT

Check out PrivacyGuides.

6. Not everything privacy focused is trustworthy

   The same level of caution that you exercise for companies which don’t value your privacy should be exercised for companies (and products) which claim to value your privacy. For example, some VPN companies would have you thinking their product is simply the best you can get when it comes to digital privacy, hell, the product is SO good, it protects you against things which are way beyond the scope of a VPN!

   Even beyond the obvious VPN example, you should extend this to self-hosted tools too! Have you reached this part of the article with the growing sentiment of "damn, maybe i should uninstall that pesky fitness app from my phone" and you are slowly warming up to the idea of cloning the repo off GitHub for the project I made? Why?

   Just because I claim to care about my privacy? Do I? How can you trust me? Are you going to download that server binary and run it on your machine without first checking if it does anything malicious? Does it handle your data well? Would you trust it to be an open directory on the Internet with little threat detection?

   Privacy is not security.

I will not plug the application I made

I don’t want this article to get reduced to don't use thing, use thing-but-made-by-me as that was not the point.

7. Avoid privacy washing

Privacy washing is a real thing. It’s so, so real that I’m putting this tip in a separate header.

I, seriously, can’t think of a single company that doesn’t, in some shape or form, say

Your privacy is important to us!

And it is! It absolutely is! Up until you read on the news that maybe they didn’t care about it enough. But fret not! They have published a public statement in which they say "hey, we goofed up, soz. we'll do better." so nothing to worry about.

Not every company does it maliciously. Very few do and even for them, you could say the intentions are muddy.

Do you care about the intentions if the result is the same?

Conclusion

Be skeptical until proven otherwise. And then be skeptical some more. Do not fall for convenience.

If you opt-out of everything you can, it will become harder^(hopefully) for companies to push surveillance systems in your everyday life.

Exercise your rights, defend your privacy, before someone deems that you never really cared about it in the first place and removes the opportunity for you to opt-out.

Is that one sad way to live? It sure as fuck is. But what other option do you really have?