Requests and parameters

run, with and make

Three static entry points reach the same instance, mirroring edulazaro/laractions so the two packages feel identical.

run(...$params) sends its arguments to handle(). Positional, named, or an associative array mapped by name:

BikeScraper::run($url);                 // -> handle(string $url)
BikeScraper::run(url: $url);            // named argument
BikeScraper::run(['url' => $url]);      // assoc array mapped by name

A single array passed to a single array-typed parameter is forwarded whole as that argument, the attribute bag:

class PriceApiScraper extends Scraper
{
    protected function handle(array $ids): array { /* ... */ }
}

PriceApiScraper::run([1, 2, 3]);        // $ids = [1, 2, 3]

with(...$params) injects into the scraper's properties by name, returns a chainable wrapper, and then you call run():

BikeScraper::with(driver: 'http', tries: 5)->run($url);
BikeScraper::with(['driver' => 'http', 'retryDelay' => 10])->run($url);

The distinction is worth internalising: run() feeds handle(), with() feeds the object. A URL goes in run(). A proxy goes in with().

make(...$deps) builds the instance through the container without running it. You rarely need it: run() already resolves through the container, so constructor dependencies are injected either way. Use it when you must drive a manually constructed instance:

BikeScraper::make($dep)->handleToResponse([$url]);

HTTP verbs, bodies and cookies

The http driver can send any verb, a request body and cookies, which covers JSON and form APIs and session-protected endpoints. These are chain methods on $this->scrape($url):

protected function handle(string $query): ScraperResponse
{
    return $this->scrape('https://example.com/search.action')
        ->driver('http')
        ->method('POST')                                   // or ->post()
        ->body(['q' => $query, 'page' => 1])
        ->asForm()                                         // or ->asJson()
        ->cookies(['JSESSIONID' => $this->sessionId], 'example.com')
        ->run();
}

These are request options, not page actions, which is why they belong to the http driver. The browser driver throws if you set method(), body() or cookies(); it navigates as a real browser instead.

Carrying a session by hand

Any Set-Cookie the server returned is on $this->request->cookies, so you can capture a session from one request and use it on the next:

protected function handle(): array
{
    $this->scrape($this->loginUrl)->driver('http')->post($this->credentials)->run();
    $session = $this->request->cookies['JSESSIONID'] ?? null;

    return $this->scrape($this->dataUrl)
        ->driver('http')
        ->cookies(['JSESSIONID' => $session], 'example.com')
        ->crawl(DataCrawler::class)
        ->run()
        ->data;
}

For a crawl of many pages, do not thread cookies by hand. Use the shared Session, which accumulates them for the whole run. See Spiders and sessions.