Exporting your Twitter bookmarks in markdown file

Featured on Hashnode

Subscribe to my newsletter and never miss my upcoming articles

People used to bookmark tweets using the like feature on Twitter. Later, they introduced Bookmarks feature that helps you save the tweets privately. And now, you have saved thousands of tweets and you can't afford to clear them.

How about we export those bookmarks safely in a file. You can use it as you like - in a web application maybe. It should be noted that Twitter DOES NOT provide an API to access the bookmarks. So, we might have to do some stuff manually here.

I am going to show you how to extract links to the bookmark as well in few lines of python code.

Step 1

Store the Twitter payload in JSON file

An alternative of this step is using the Postman Client. You might want to try it out first, here is the link to a blogpost that allows you to send the request and it's download feature enables you to download the response in a click.

If you don't have the setup, we can get back to copy pasting the response from our good old browser DevTools.

  • Open Twitter web and go to the Bookmarks page
  • Open Dev tools and go to Network tab
  • Clear the network logs and select XHR
  • Apply filter as "bookmark"
  • Refresh the twitter bookmarks page

You would see some API calls are made as bookmark.json like below image.

Screenshot 2020-11-01 at 3.22.19 PM.png

If you have tons of bookmarks, you might have to scroll till the end to get all pagination requests.

  • After reaching the end. Select each request and copy paste the response received from the Response tab as shown in the image above.
  • Save each response object with .json extension.

Step 2

  • Once you have saved the JSON files, put them in a folder.

Now we just have to write a python script that would get us the links to the Bookmarks and text of the tweet!

  • Reading JSON files from folder
import json
import glob

# using glob to read all files in the folder
files = [file for file in glob.glob("JSONBookmarks/*")]     
for file_name in files:
    print(file_name)
    with open(file_name) as bk:
        data = json.load(bk)        # reads json data
    all_bookmarks.append(data)

Here, JSONBookmarks is the folder name. glob() takes the path of the folder to read files from. /* means all files.

The twitter payload explicitly doesn't have the link to the bookmarked tweets.

It provides two primary objects of our use with keys:

  1. tweets
  2. users

We will have to use this data in the payload to construct the url to the tweet.

# Function to construct bookmarked tweet url
def constructUrl(tweet_id, username):
    return "https://twitter.com/" + username + "/status/" + tweet_id

Username can be accessed like this.

# Function to get username
def getUserName(id):
    return response["users"][id]["screen_name"]

Now the final code that loops through all the tweets and writes in the markdown

# saving in markdown file, if no file exists  using '+' creates one
md_file = open("bookmarks.md", "w+")

# Run a loop through all_bookmarks
for data in all_bookmarks:
    response = data["globalObjects"]
    for tweet_id in response["tweets"]:
        tweet = response["tweets"][tweet_id]
        text = tweet["full_text"]
        url = constructUrl(tweet_id, getUserName(tweet["user_id_str"]))
        bookmarked_tweet = "\n- " + text + "\n" + "\t - " + url
        md_file.write(bookmarked_tweet)

The final output would look like this:

Screenshot 2020-11-01 at 4.15.56 PM.png

I have formatted it as I wanted, you can do something else too. Here is the whole code for you to refer in this gist.

Other Blogposts you might like to checkout!

No Comments Yet