Reading and writing

Sources are cached in the sources table through the SourceRecord model, keyed by the model, the source name and the variant. Everything in this page is about one invariant: the row may fall behind the service, and it may never get ahead of it.

Reading serves the row

Touching an attribute autoloads. It fills from the stored row if there is one, and goes to the origin if there is not:

$city->source('weather')->temperature;   // the row if it exists, the provider if it does not

When the origin cannot answer that first read, the source builds what it can in memory and nothing is stored. A row written there would say the provider has data it has never seen, and since a stored row is trusted, nothing would ever go back to ask. The next read tries the provider again.

Only the origin's own failure is caught that way. A bug in an origin, a TypeError or a misspelled method, surfaces instead of being dressed up as a service being down.

Asking for live data

$city->source('weather')->fetch();

fetch() always goes to the origin, and always writes the result through to the row. There is no flag to turn that off. A cache is the local picture of something remote: a live read that leaves the old copy behind turns the next screen into a lie and nobody gets told.

No TTL

What the package will not do is expire that cache on its own. Once the row exists, reading attributes never calls the origin again, and you refresh when your application decides to: on a schedule, on a button, when a webhook says something changed.

A TTL that fires on attribute access sounds handy until a listing page paints twenty cards and makes twenty API calls, inside the request, in series.

The row

if ($source->record()) {
    // it has been fetched or saved at least once
}

record() looks the row up in the table the first time it is asked for, so it answers "is this stored?" even on a source that has not read anything yet, and it never calls the origin. A miss is remembered, so asking twice in a loop does not query twice.

Comparing both pictures needs nothing special:

$stored = $source->record()->toArray();   // name, variant, origin, attributes, dates
$live   = $source->fetch()->toArray();    // the fields, straight from the service

And a row turns back into its source, which is how a scheduled command works through a set of them:

$source = $record->toSource(ListingSource::class);

The variant and the row travel with it, so persisting from there updates the row it came from instead of writing a second one next to it.

The signature

Every row carries an md5 of its payload. It is what tells you whether anything actually changed between two reads without comparing JSON by hand, and it earns its keep when the next step costs money:

$before = $source->record()->signature;

$source->fetch();

if ($source->record()->signature !== $before) {
    TranslateListing::dispatch($source->record());
}

Writing

$property->source('listing')->save();

save() builds, pushes, and writes the row after the push returns. That order is the whole design: when the push fails the row is not touched, so the local copy can fall behind the service, which the next write or read fixes, but it never gets ahead of it. A row claiming something exists on the other side when it does not is the failure you never recover from, because nothing goes looking for it again.

Failure arrives as an exception, which is what a queued job wants:

public function handle(): void
{
    try {
        $this->property->source('listing')->setVariant($this->operation)->save();
    } catch (OriginException $e) {
        $this->property->updateQuietly(['listing_errors' => $e->getMessage()]);

        $this->release($this->backoff[$this->attempts() - 1] ?? 300);
    }
}

When the caller renders instead of retrying

$result = $property->source('listing')->trySave();

if ($result->failed()) {
    return back()->withErrors($result->message);
}

trySave() reports the outcome instead of throwing, and persists nothing unless the origin took the data. The result carries the reason, which a boolean could not. Use it where a 500 would be the wrong answer: a Livewire component, a controller, an Artisan command printing a table.

What the origin reports

Returning the response array means Saved. When a service takes the payload and finishes with it later, the origin says so:

use EduLazaro\Larasources\Enums\OriginStatus;
use EduLazaro\Larasources\OriginResult;

public function save(array $data): array|OriginResult
{
    $response = $this->http()->post($this->endpoint, $data)->throw()->json();

    return new OriginResult(
        status: $response['published'] ? OriginStatus::Saved : OriginStatus::Processing,
        data: $response,
        message: $response['reason'] ?? null,
        externalId: $response['id'] ?? null,
    );
}

match ($result->status) {
    OriginStatus::Saved => $this->notify('Published'),
    OriginStatus::Processing => $this->notify('Publishing'),
    OriginStatus::Failed => $this->notify($result->message),
};

OriginStatus has three cases and only two of them are ever stored: a row exists because an origin took the data, so it reads saved or processing, never failed.

Resolving what is still processing

A processing row is not a lie and not the truth yet. Reading it never calls the origin, because a listing page is not the place to discover that forty rows each need an API call. What is pending is a query:

use EduLazaro\Larasources\Models\SourceRecord;

foreach (SourceRecord::processing()->where('updated_at', '<', now()->subHour())->get() as $record) {
    try {
        $record->toSource(ListingSource::class)->reconcile();
    } catch (OriginException $e) {
        Log::warning('Could not reconcile ' . $record->getKey() . ': ' . $e->getMessage());
    }
}

reconcile() re-reads from the origin while the row is still processing, and does nothing when it is not. It is the only thing that turns it into saved, and it converges on whatever the service really ended up with, even when the payload was rejected in the end. A row that stays processing for too long is itself the signal that something got stuck.

Deleting

$property->source('listing')->delete();

The origin deletes, and the row goes only if it did. A failure propagates and the row stays. An origin that returns a literal false is saying it deleted nothing, so the row stays for that too: clearing it then would leave your database claiming the resource is gone while the service still has it.

Deleting something that is already gone is not a failure. An origin that gets a 404 should return normally, which is what the bundled RemoteOrigin does.

$property->source('listing')->clear();   // drop the local row, leave the service alone