Recently I came across the following tweets from an SEO expert called Luke Davis:
Seriously bummed me out. It deleted a few other lists out of nowhere but that was really important to me. I hate this place.
— Luke Davis (He/Him) (@LukeDavisSEO) August 19, 2020
I make extensive use of Twitter lists myself so I could feel his pain. And worryingly, I was able to find lots of other Twitter users whose lists had suddenly disappeared. For example:
@TwitterSupport where did all my Twitter list go? Most have been deleted. Took years to create now they're gone with no reason
— Rodney first, then Littles (@Littles1126) July 31, 2020
Twitter wasn't always to blame - sometimes it was just a case of good old-fashioned user error:
I accidentally deleted my favourite twitter list. pic.twitter.com/DBrLoHcOHr
— M U K I (@bwanamuki) July 14, 2020
A few days before I saw Luke's tweet, my application for a Twitter developer account had been approved. This sounds grand but basically meant I was allowed to use the Twitter API. I wondered: would it be possible to write something in Python that used the Twitter API to solve Luke's problem - by letting users back up their Twitter lists?
Despite my limited Python knowledge, the short answer is yes. I wrote a script that lets you specify both a Twitter account and the name (slug) of a public list belonging to that account, then export a text file (.txt) of all users in that list. The text file is saved to your desktop with the filename listname.txt. So for example a list called 'SEO' would give you a file called 'seo.txt'.
Before I share the code below, I'd just like to run through a few caveats:
The script won't run 'as is' because it needs keys and tokens. Specifically these four values need to be added to the script where I've indicated:
To get hold of these you will also need your own Twitter developer account I'm afraid. This is free, but you will need to go through the application process.
The script uses the Python package Python Twitter Tools so you will need to install this. It's available on the Python Package Index, aka PyPI: https://pypi.org/project/twitter/
The script uses the Twitter 1.1 API. In August this year, Twitter's new v2 API went live in early access. The new API does not yet have any lists functionality (hence me using 1.1) but it is on the roadmap for it to do so. The Twitter developer blog says that "Eventually, the new API will fully replace the v1.1 standard, premium, and enterprise APIs", at which point I presume my script will need to be updated to use the new API instead.
My script does not have any error handling. If, for example, you enter a name of a list that does not exist, you will get a nasty (if ultimately harmless) error and the script will terminate. I have, however, applied some formatting to the input to make it a bit less strict: the list name is converted to lower case and any spaces are converted to hyphens, to match the API's expectations.
Conversely nothing needed to be done around account name (aka Twitter handle), as these are not case-sensitive and the API accepts them both with and without a preceding '@'.
The script will happily overwrite an existing file on your desktop. So if you have a file called important.txt and then export an unrelated Twitter list called 'Important', you can say goodbye to your original document. Sorry!
More professional solutions undoubtedly exist. For example I've come across a Chrome extension called Twlets which lets you export all sorts of Twitter data, including lists, to Excel. But while I haven't tried Twlets myself I see that it is only "free for the first 1,000 items exported". Conversely my solution, while entirely amateur, has enabled me to export a list of 4,400 members completely free of charge.
All that said, here's the code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | #Install Python Twitter Tools package before starting: https://pypi.org/project/twitter/ #Paste in your key and token values below api_key="" api_secret_key="" access_token="" access_token_secret="" from twitter import * import os twitteraccount=input("Which Twitter account do you want to export from? ") twitterlist=input("What list do you want to export? ").lower().replace(" ","-") myfilepath=os.path.expanduser("~/Desktop/")+twitterlist+".txt" t = Twitter(auth=OAuth(access_token, access_token_secret, api_key, api_secret_key)) mydic=t.lists.members(owner_screen_name=twitteraccount, slug=twitterlist, count=5000) myfile=open(myfilepath,'w') myfile.close() myfile=open(myfilepath,'a') for x in range(len(mydic['users'])): myfile.write("@"+mydic['users'][x]['screen_name']+"\n") myfile.close() print("Done! See "+myfilepath) |
A sensible next step would be to write a script that enables you to create a Twitter list from an exported .txt file. That would allow you to restore a list that has gone missing, or indeed copy an existing list (either one of your own or someone else's). I may get round to this in future, but if this sounds like a fun project then maybe start by looking at the Twitter API lists documentation.
I hope this has helped any Twitter users or, more likely, budding Pythonistas out there.
