redux-cheerio
Cheerio middleware for Redux
Motivation
Dispatch a Redux action and get back pretty JSON response.
Cheerio works under the hood for parsing the HTML or XML document of HTTP requests. Cheerio uses a very simple, consistent DOM model. As a result parsing, manipulating, and rendering are incredibly efficient.
Click here for more info on Cheerio
Installation
Guide
Configure your store with the middleware
Insert redux-cheerio into the middleware chain like you would with any other piece of middleware.
Go to the end of this readme for a full self-contained example of using redux-cheerio
Usage
Dispatch actions to your Redux store that have a type of CHEERIO_TASK and a payload consisting of a url to make the request to and a task function whose job is to parse the HTML using jQuery selectors.
To use the middleware, dispatch an action takes the following form:
#Example
Here is an example action that returns the HTML body of a response as a JSON object.
Define the first Cheerio action which makes the request
Note that in this example we set the following custom payload properties:
A url
Where the HTTP request will be made to.
A function that allows us to use Jquery selectors such as '$(div).text()'
Under the hood our middleware takes the HTML from the response and calls the Cheerio.load function like so
Now we can use jQuery selectors to extract the the data we want from the HTML. This is especially useful for webscraping. We must remember to return the result of this selection.
Dispatch the first action
Lets dispatch the action to make the request and parse the response with the task function in our action.
Now watch as redux-cheerio handles the dispatching of one further action depending on the success of the HTTP request.
redux-cheerio middleware dispatches an additional fulfilled action if the request was successful
When our task function returns something a CHEERIO_TASK_FULFILLED action will be dispatched as long as no errors have occured. The payload of this new action will consist of the thing we returned in our task function that was in the first CHEERIO_TASK action.
redux-cheerio middleware dispatches an additional rejected action if the request was not successful
If there was an error during the request such as a timeout or 404 status code then redux-cheerio middleware will dispatch a rejected action instead of a fulfilled one.
Full example
An aside - Webscraping
Flux implementation is a useful mental model for webscraping.
The webscraping model using cheerio middleware
Actions are attempts to extract data from the outside world
These actions return raw information about the outside world to the reducers
Reducers then extract the valueable information from this raw data and place it in the store
We can now dispatch more informed actions based on the data we extracted in the previous round
Scrape in a declarative manner in Redux. Just create a cheerio action with a url and a function representing a scraping task.
Decompose complex web scraping workflows into discrete redux actions that represent small scraping tasks.
Dispatch an action with a jquery selector and a url and the resulting data or error will be sent through the rest of your middleware and end up in your reducer.
Therefore the state of your app can be modified according to the state of other websites.
Example uses for webscraping
logging other websites and tracking given selectors over time
keeping your app in sync with data from websites that do not have an api
Last updated