Automating Keyword Research and Competitor Analysis with AI and n8n

When doing keyword research or competitor analysis, we tend to take more and more time trying to get things done. Automating this process can be on eof the most effective strategies in which we can be able to use AI to analyse competitor posts, get relevant insights and generate a strategy we can use to outrank them

Keyword researrch is key for most of the SEO efforts that we are currently running or managing. For busy entrepreneurs, freelancers or marketers, undertaking this whole process of keyword research, Tracking, getting the competitor website and checking other metrics can take quite some time.

We can be able to 10X our productivity and results by integrating AI into this process,

This was the reason we built this tutorial, to show you how you to can build and automate your keyword resrearch process

What we need to automate our keyword research

For this process,

1. we are going to automate Keyword research using gogle Gemini, an AI model with various capabilities and uses(model used is gemini-1.5-pro-latest). Any other model can be used
2. This data shall be built on airtable for easy visualiations. Those on pro/Team plan can use the scripts to send post requests emmediatly ne data is added
3. N8n to automate this whole process
4. API key for your keyword research tol. In this case we shall be using the Effibotics Keyword tracking app
5. A copy of this airtable document. You can format the tabe to your desired format

Getting started

The first step of our process will be creating a http module to the Effibotics Keyword tracking tool, that we shal be using for two purposes

1. Get new keyword ideas that we need to generate keywords for
2. Get the search volum eof the keywords

This can be easily done using chrome dev tools while you are logged in

Next we are going to set up our project to on airtable by adding the airtable base to our database

Automationg the keyword research

You can set your workflow to start on a cron/ Schedule, ie every hour, day,. week, month etc or you can do it manually by triggering one keyword at a time

To trigger manually, we will use the Keyords table to update the status we need to check. When the result if found, we will have the results on the next table

Press enter or click to view image in full size

Once we have everything settled, we are going to get started on our automation workflow.

Press enter or click to view image in full size

The setup here includes getting and cleaning up the results from Effibotics Keuword tracking app to a more simplified format

If there is an error on the workflow it gets updated on the airtable with the respective error message and an update is made that the keyword was generated.

Getting the keyword positions on Google

Once we have the new keywords generated, we can go a step further and get their positions on SERP based on the differnt geographical locations.

You can be able to do this with a single API call to the refresh endpoint and get the top 100 keywords that are ranking for the respective keywords.

Press enter or click to view image in full size

This is a critical step for identifying competitors who are ranking for a certain keyword.

Once we get these results we can be able to manipulate these results to get more detailed bsed n thecontent the wensite is ranking for

Press enter or click to view image in full size

Setting up the AI automation

Once w have the keywords, page and website that are ranking for the website, we ar egping to send a http request, get the page data and sensd it to an AI for further analysis and detaining. The AIs that an be used here can be Gemini, OpenAI, claude, Ollama among others

Press enter or click to view image in full size

Set and finetune the prompt on the AI agent to fit your desired settings in order to get the bet results. The more specific a prompt is, the better the guidance or advice that wil be provided for your SEO strategy

Updating keywords to Airtable

Once we have the response, we are foiung to do a few manipulations to set the text format and the upload the data to airtable

The keywords are updated on a diffent table with all the actionable SEO strategies provided by the AI, and an update made on the main table

Press enter or click to view image in full size

Each of the above keyods contain a strategy based on the competitors website copy that ranks for the specific keyword

Here is an example of the SEO strategy for the different keywords and websites

https://youtu.be/NOTkkVtiWmY

Below is a strategy provided for the keyword marketing and digital marketing on the website mailchimp

Conclusion

Once you have the above data, you can derive real and actionable steps and strategies that will massively improve your efficiency, productivity and results