← Larascraper 3 / 17

Your first scraper

Your first scraper

Generate the class

php artisan make:scraper BikeScraper

That writes app/Scrapers/BikeScraper.php:

namespace App\Scrapers;

use EduLazaro\Larascraper\Scraper;
use EduLazaro\Larascraper\Support\ScraperResponse;

class BikeScraper extends Scraper
{
    protected function handle(string $url): ScraperResponse
    {
        return $this->scrape($url)->run();
    }
}

Two things are happening, and keeping them apart is the whole mental model:

  • BikeScraper::run(...) is the entry point from the outside. It resolves the scraper through the container and calls handle() with whatever you passed.
  • $this->scrape($url) is the fetch chain, and it lives inside handle(). You configure the request on it and finish with a terminal.

The fetch chain

$this->scrape($url) returns a builder. You chain configuration and page actions on it, and a terminal decides what comes back:

Terminal Returns
->run() A ScraperResponse. Its data is the raw HTML, or the Crawler's parsed data when you chained crawl().
->text() The text of the first match, when crawl() was given a CSS selector.
->texts() A string[] of every match, same case.
->file() A CapturedFile, for binary responses and captured downloads.
->fetch() The RequestResponse itself, when you want the transport layer and nothing else.

A bare ->run() gives you the page:

$html = BikeScraper::run('https://shop.com/bikes/4')->data;

Parse it with a Crawler

A Crawler is a plain class that only knows about documents:

namespace App\Scrapers\Crawlers;

use EduLazaro\Larascraper\Crawler;

class BikeCrawler extends Crawler
{
    protected function handle(): array
    {
        return [
            'name'  => $this->filter('h1')->text(''),
            'price' => $this->filter('.price')->text(''),
            'specs' => $this->filter('ul.specs li')->each(fn ($li) => trim($li->text(''))),
        ];
    }
}

Chain it and the response's data is whatever the Crawler returned:

protected function handle(string $url): ScraperResponse
{
    return $this->scrape($url)
        ->crawl(BikeCrawler::class)
        ->run();
}

$result = BikeScraper::run('https://shop.com/bikes/4');

if ($result->success) {
    $bike = $result->data;   // ['name' => ..., 'price' => ..., 'specs' => [...]]
}

When a class is too much

For one or two values, pass a CSS selector to crawl() instead of a class and finish with an inline terminal:

protected function handle(string $url): array
{
    return $this->scrape($url)
        ->crawl('.bike-card h3')   // a selector, not a class
        ->texts();                 // string[] of every match
}

handle() returned a plain array there and never mentioned ScraperResponse. That is fine: run() wraps whatever handle() returns. See Responses and failures.

The shortest possible scraper

Putting it together, this is a complete, working scraper:

class TitleScraper extends Scraper
{
    protected function handle(string $url): array
    {
        return $this->scrape($url)->crawl('h1')->texts();
    }
}

$titles = TitleScraper::run('https://example.com')->data;   // string[]

That is the whole loop: a Scraper to fetch, a Crawler to parse, a ScraperResponse to read. Spiders, captchas, PDFs and Fibers all build on exactly this.

Where to go next

The page you are scraping decides which chapter you need:

PreviousInstallation NextDrivers