Sarthak's blog
Automate Your Job Search: How I Built a Python Selenium Bot to Fetch LinkedIn Jobs in Seconds

Automate Your Job Search: How I Built a Python Selenium Bot to Fetch LinkedIn Jobs in Seconds

Sarthak Thakkar's photo
Sarthak Thakkar
·

3 min read

Introduction

If you've ever been on the hunt for a job, you know firsthand the time and energy it takes to examine countless job descriptions and submit applications. As a fellow job seeker, I experienced this frustration myself.

Recognizing the need for a better solution, I decided to create a LinkedIn Job Bot to streamline the process. This powerful tool hunts job postings on LinkedIn for positions that match your preferences, and conveniently stores all relevant data in a CSV file for your review. Plus, you can easily run the bot multiple times and the data will be saved in a new file each time.

Save yourself the headache of endless job searching and let my LinkedIn Job Bot do the heavy lifting for you.

Methodology

At the core of this project lies the powerful automation tool known as Selenium. With its open-source capabilities, Selenium allows us to automate and extract data from almost any website on the internet. Supporting Python libraries such as Pickle and CSV further enhances the tool's functionality and enables efficient data serialization and management.

To get started with this tool, you'll first need to download a Chrome web driver that Selenium can recognize and operate with. The complete source code, including a detailed README file, can be found here (Source code).

To make the most of this tool, simply input your desired data into the my_data.json file and run the main.py file to see the magic happen before your eyes.

Results

After running the code, the results are stored in the outputs folder, containing crucial information such as:

  • Job Role

  • Company Name

  • Job Location

  • More Info

As with any website, LinkedIn undergoes frequent updates that may require modifications to the XPATHs within the codebase. However, once the data is extracted, we can easily sift through and identify desirable job opportunities, streamlining the job application process.

Technical Details

One of the fundamental principles incorporated into the development of this bot is the utilization of Pickle to save client-side cookies. This enables the user to log in seamlessly without having to re-enter credentials repeatedly. Notably, such an approach is critical in avoiding any potential account lockouts resulting from numerous login attempts. The code implementation follows a similar structure to the following example.

def save_cookies(self):
    """Cookies saved so that login does not occur every time."""
    with open(self.PICKLE_FILE, 'wb') as f:
        pickle.dump(self.driver.get_cookies(), f)
        logging.info("Cookies saved to Local")

def fetch_cookies(self):
    """Fetch the saved cookies."""
    with open(self.PICKLE_FILE, 'rb') as f:
        cookies = pickle.load(f)
        for cookie in cookies:
            self.driver.add_cookie(cookie)
        logging.info("Cookies fetched by browser")

Furthermore, to ensure effective debugging and error correction, I also incorporated logging functionalities into the code. Additionally, to enhance code readability and ease of understanding, the code also includes detailed docstrings that provide an insight into the workings of the code.

Conclusion

To conclude, the development of a LinkedIn bot with Python and Selenium can significantly simplify the job search process. By integrating other Selenium tools, we can further enhance the functionality of the code, and I welcome any suggestions for improvement. If you have any ideas that you would like to see integrated into the bot, please raise an issue for it, and we will work towards making it better. The possibilities are endless, and by leveraging the power of automation, we can make a small difference.

Thank you for reading! This is Sarthak and until next time, peace out!

Pythonautomationjob searchjobsLinkedIn