Exporting Twitter lists using Python

13 September 2020

Recently I came across the following tweets from an SEO expert called Luke Davis:

I make extensive use of Twitter lists myself so I could feel his pain. And worryingly, I was able to find lots of other Twitter users whose lists had suddenly disappeared. For example:

Twitter wasn't always to blame - sometimes it was just a case of good old-fashioned user error:

A few days before I saw Luke's tweet, my application for a Twitter developer account had been approved. This sounds grand but basically meant I was allowed to use the Twitter API. I wondered: would it be possible to write something in Python that used the Twitter API to solve Luke's problem - by letting users back up their Twitter lists?

Despite my limited Python knowledge, the short answer is yes. I wrote a script that lets you specify both a Twitter account and the name (slug) of a public list belonging to that account, then export a text file (.txt) of all users in that list. The text file is saved to your desktop with the filename listname.txt. So for example a list called 'SEO' would give you a file called 'seo.txt'.

Before I share the code below, I'd just like to run through a few caveats:

You need keys and tokens

The script won't run 'as is' because it needs keys and tokens. Specifically these four values need to be added to the script where I've indicated:

  •  API key
  • API secret key
  • Access token
  • Access token secret

To get hold of these you will also need your own Twitter developer account I'm afraid. This is free, but you will need to go through the application process.

You need Python Twitter Tools

The script uses the Python package Python Twitter Tools so you will need to install this. It's available on the Python Package Index, aka PyPI: https://pypi.org/project/twitter/

The API may get deprecated

The script uses the Twitter 1.1 API. In August this year, Twitter's new v2 API went live in early access. The new API does not yet have any lists functionality (hence me using 1.1) but it is on the roadmap for it to do so. The Twitter developer blog says that "Eventually, the new API will fully replace the v1.1 standard, premium, and enterprise APIs", at which point I presume my script will need to be updated to use the new API instead.

There's no error handling

My script does not have any error handling. If, for example, you enter a name of a list that does not exist, you will get a nasty (if ultimately harmless) error and the script will terminate. I have, however, applied some formatting to the input to make it a bit less strict: the list name is converted to lower case and any spaces are converted to hyphens, to match the API's expectations.

Conversely nothing needed to be done around account name (aka Twitter handle), as these are not case-sensitive and the API accepts them both with and without a preceding '@'.

It likes to overwrite files

The script will happily overwrite an existing file on your desktop. So if you have a file called important.txt and then export an unrelated Twitter list called 'Important', you can say goodbye to your original document. Sorry!

There are alternatives

More professional solutions undoubtedly exist. For example I've come across a Chrome extension called Twlets which lets you export all sorts of Twitter data, including lists, to Excel. But while I haven't tried Twlets myself I see that it is only "free for the first 1,000 items exported". Conversely my solution, while entirely amateur, has enabled me to export a list of 4,400 members completely free of charge.

All that said, here's the code:

A sensible next step would be to write a script that enables you to create a Twitter list from an exported .txt file. That would allow you to restore a list that has gone missing, or indeed copy an existing list (either one of your own or someone else's). I may get round to this in future, but if this sounds like a fun project then maybe start by looking at the Twitter API lists documentation.

I hope this has helped any Twitter users or, more likely, budding Pythonistas out there.

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
James Clark
Hi! I'm James Clark and this is my website. Need help with your own website's analytics, SEO, content or ads? Contact me
Copyright © James Clark
0
What do you think? Leave a commentx
linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram