Files and PDFs

When the response is a file rather than a page, you grab it in the chain with capture() and finish with the ->file() terminal, which runs the fetch and returns a CapturedFile.

From a click or a link

use EduLazaro\Larascraper\Scraper;

class ReportScraper extends Scraper
{
    protected function handle(string $pageUrl): array
    {
        $file = $this->scrape($pageUrl)
            ->click('a.download-pdf')
            ->capture('application/pdf')   // grab the response the click triggers
            ->file();                      // run the chain, return the CapturedFile

        $file->save(storage_path('app/report.pdf'));

        return ['bytes' => $file->size()];
    }
}

capture($expect) records the file-like responses the page produces and keeps the one matching expect, a content-type substring such as application/pdf. PDFs also match by their %PDF magic bytes. With no argument it takes the first file-like response.

The ->file() terminal returns the CapturedFile, or raises a no_file scrape failure, a ScrapeException folded into success = false, when nothing was captured.

It captures files that render inline, the browser's built-in PDF viewer being the usual case. A forced download, Content-Disposition: attachment, is not captured.

From a form

When the file only comes back from submitting a form, with its hidden fields, tokens and session, submit and capture:

->submit('form')->capture('application/pdf')->file()

submit() plus capture() is the composable pattern. The old one-call submitAndCapture() is deprecated.

From a direct URL

If the file lives at a plain URL you do not need a browser at all. The http driver downloads it directly, and a binary response is exposed through the same ->file() terminal, and on $this->request->file. Text responses (HTML, JSON, XML) still arrive as the response html.

class LawScraper extends Scraper
{
    protected string $driver = 'http';

    protected function handle(string $url): string
    {
        return $this->scrape($url)->file()->text() ?: '';
    }
}

$text = LawScraper::run('https://example.com/law.pdf')->data;

Behind a captcha

Wrap the capture in repeatUntil(Condition::captured(), ...), solve the captcha first, and re-navigate each attempt when the site regenerates state:

use EduLazaro\Larascraper\Support\Condition;

$file = $this->scrape($viewerUrl)
    ->repeatUntil(
        Condition::captured(),
        fn ($b) => $b
            ->visit($viewerUrl)                                           // fresh state each try
            ->gotoAttr('object[type*="pdf"], embed[type*="pdf"]', 'data') // real URL and fresh captcha
            ->when(
                Condition::selectorExists('img[src*="captcha"]'),
                fn ($c) => $c->solveCaptcha('img[src*="captcha"]', 'input[name=captcha]'),
            )
            ->submit('form')
            ->capture('application/pdf'),
        max: 8,
        delay: 400,
    )
    ->file();

gotoAttr() is what handles a viewer that never exposes a link: the real document URL lives in an <object data="…"> attribute, so there is nothing to click.

Reading a PDF

$file = $this->scrape($url)->click('a.pdf')->capture()->file();

$text = $file->text();          // the PDF's text layer: fast, free

if ($text === '') {             // scanned PDF, no text layer
    $text = $file->vision('ai');
}

The CapturedFile API:

Try text() first, always. It is free and instant, and most PDFs published by an institution have a text layer. vision() costs money per page and should be the fallback, not the default.

System requirements

Each engine shells out to its tool, or for vision('ai') calls an OpenAI-compatible endpoint. A missing binary fails with a clear message. These are OS packages, so Composer cannot install them; add them to your image.

vision('ai') still needs poppler-utils: it rasterizes each page with pdftoppm before sending it to the model.

Configure the ai engine through config/larascraper.php with openai_key, vision_model, vision_lang and vision_dpi, or through the OPENAI_API_KEY environment variable.

A complete example

use EduLazaro\Larascraper\Scraper;
use EduLazaro\Larascraper\Support\ScraperResponse;

class LawScraper extends Scraper
{
    protected function handle(string $pageUrl): ScraperResponse
    {
        $file = $this->scrape($pageUrl)
            ->click('a.download-pdf')
            ->capture('application/pdf')
            ->file();                       // CapturedFile, or a 'no_file' failure

        $text = $file->text();

        if ($text === '') {
            $text = $file->vision('ai');
        }

        return trim($text) === ''
            ? $this->fail('no_text')
            : $this->ok(['text' => $text]);
    }
}

The caller gets 'no_file' when nothing was captured and 'no_text' when the document was empty, which are different problems and worth telling apart.