Gather Client Information From LinkedIn Using Python

One of the biggest challenges for any marketer is to create content that is relevant to their target market. Because content is the basis of all marketing today, getting it right is very important. Have the wrong content, and you repel the people you’re trying to attract. But have the right content, and you’re golden.

Since becoming the Director of Engagement at Intridea, I’ve been working to answer this question for us. The pursuit of an answer has led me to begin gathering information from a variety of sources in order to get a full picture of what’s “relevant” to our clients. One of those sources is LinkedIn.

Scraping Data From LinkedIn With Python

A short time ago I open sourced one of the scripts I created to gather company information from LinkedIn: LinkedIn Data Miner.

In the first version, given a MongoDB database with client information including the url of their LinkedIn company page, the script will go to LinkedIn, scrape the page, and update their record with the following:

1. Description
2. Specialties
3. Address
4. Website
5. Company Type
6. Founded
7. Industry
8. Company Size

Once you have the data in your database, you can do a lot with it, including:

1. Creating simple charts in Excel or Numbers
2. Mashing it up with other client information and visualize it in Tableau
3. Add it to the client data you already have in your CRM

The point is to get the information out of LinkedIn and with the rest of your client data.

What You Can Do With This

One of the reasons I created this script was that I wanted to see what industries our clients were in. Once I had scraped the data I created a tree map in Tableau:

The darker and larger the square, the more client records I had for that industry. And yes, my chart has labels on it.

A few other charts I created were:

1. Client map: see how many clients we had in each state, on a map
2. Clients by company size: another tree map like the one above, only showing how many clients we have of each size (per LinkedIn)
3. Client industries by state: how many clients we have in each industry, in each state

All of this is the tip of the iceberg. I’ve also created some python scripts to get data out of Google Analytics. The real power comes in when you start to mash up data from all of these sources.

Your Turn

Now it’s your turn to scrape LinkedIn. Go ahead and install MongoDB, get your client records in there along with their LinkedIn company page URLs, download the LinkedIn Data Miner script, and go to town!

Let me know how it works for you.