GSoC 2018 - Week - 1

Ankit Jain
Ankit Jain DevOps Engineer @Razorpay | CommunityBridge Mentee @Keptn | @Drupal GSoC Alum | #GitHubCampusExpert Alum | GCI & GSoC'19 Mentor @Drupal | Speaker | Writer

GSoC 2018 - Week - 1

Coding Period is started and I am very excited to start working on my project - Affiliates Connect. This module provides an interface to easily integrate with various affiliate APIs or product advertising APIs provided by different e-commerce platforms like Flipkart, Amazon, eBay etc to fetch data easily from their stores and display those to monetize your website by advertising their products.

Project Roadmap

This project is divided into three phases -

1st phase deliverables -

Fully functional affiliates_connect module with all required tests and API documentation for developers.

2nd phase deliverables -

3 custom modules affiliates_connect_amazon, affiliates_connect_flipkart, affiliates_connect_ebay to integrate with Amazon, Flipkart and eBay respectively

3rd phase deliverables -

1. Cloaking and hit analysis feature to let site builder checks his marketing/advertising reports.
2. Form widget browser to let site builder creates advertising content.
3. Product Search Page for side by side comparison of products from different vendors.
4. A Drupal distribution of product compare engine
5. Documentation and screencast explaining how to use this module.

Project Roadmap issue link - Project Roadmap

Read more about the phases under the child issue.

1st phase issue link - Affiliates Connect - Phase 1

2nd phase issue link - Affiliates Connect - Phase 2

3rd phase issue link - Affiliates Connect - Phase 3

Project management

Project management is done through project issue queue, I will create an issue for each task and will attach the patches so that mentors can easily review my work and get to know my progress.


This module has two main functionalities i.e -

1. It provides developers to leverage its functionalities by providing their own plugins e.g. affiliate_connect_example module to fetch data from example website.

2. It provides scraper as a fallback fetcher

In this week, I am working on creating database schemas for storing the product’s metadata and user’s configuration for various modules/plugins. A schema is designed using Custom entity.

After finishing the database schema, I will start working on the scraper API. Scraper API basically scrape the data from various e-commerce sites. For scraping, I am using Node.js and provide REST API for each of the scraping functions. HTTP request will be sent from Drupal through cURL or PHP Guzzle package to Node API which will call the respective function for scraping and sent data back to Drupal as JSON response.

Now the question arises, why Nodejs? why not any other language?

This is because I find x-ray module very appropriate for this project as it has inbuilt pagination feature, plugin system and supports phantomjs for dynamic scraping, delays, throttle, timeout and many more feature. One of the other reason for using Node.js is because Drupal is adopting some frameworks which are powered by Node.js like Nightwatch for functional browser testing. Here is the link to the post on Drupal Planet regarding the same - Nightwatch in Drupal Core

Another question is, why REST API architecture? Why not XML-RPC or SOAP?

REST is one of the most popular ways of making Web Services work. By providing REST API, Drupal can interact with the scripts by sending HTTP request and exchange data in JSON format. REST is now the Drupal Standards.


  • Most of the websites are using latest Front-end Frameworks like Angular, React e.t.c so to fetch dynamic content from such websites, we need to use a Headless browser for which I am using x-ray in Node.

  • My end semester exams are going on which will end on 24th of May so I won’t be able to work with my full potential, I will try to work as much as I can during this interval and will cover my work after my exams positively.

comments powered by Disqus