Skip to content
Scalpels home

Sign up now for $10 in credit on a new Laravel Cloud account.

Idea

A central log sink for many Laravel apps

· Ed Grosvenor

Most of the organizations we work with run more than one Laravel application. Some run a lot of them. Their logs live wherever each app happens to write them. That might be a file on an instance that is gone after the next deploy, a hosted service one app was wired to years ago, or a channel nobody has looked at since it was configured. Asking a question that spans two of those apps means opening two things. Asking one that spans forty is not a question anybody asks.

So we want something like Papertrail for an organization with a lot of Laravel apps. Each app installs a log driver. The driver ships that app's log records to one ingest API. Every app's logs land in the same place, and the organization's agents explore all of them through a single MCP server. There is no per-app dashboard and no end-user accounts, just an admin UI and the MCP.

We know someone running about forty Laravel apps who would love one MCP that lets them talk to all of their logs. That is the whole product idea, and they are the reason we think it is worth an experiment rather than a conversation.

The experiment repository is artisan-build/ducklake-logs-experiment. It is temporary. We are finding out whether this can be built, not building it.

Raw by default

The driver sends log records unmodified. Whatever the application wrote, the sink receives.

That is a deliberate choice, and it has an obvious cost. Applications log things they should not, and once logs leave the app, there is one more place holding them. The alternative is to quietly drop fields and have somebody discover a year later that the one line they needed was filtered out before it was ever sent.

Redaction is a hook. The payload filter is a rebindable binding that the application overrides. An app that needs to strip a header, a token shape, or a customer field does it in its own code, where the people who know what is sensitive are. We would ship guidance written for an agent to follow. The app's own agent could then write that filter, instead of the person installing the driver reading a reference page and guessing.

Why not Postgres

Hone already collects telemetry from a team's Laravel apps and keeps it in the deployment's own Postgres database. It works because it does not keep much raw data. Individual events live for 72 hours by default. Hone rolls them into daily aggregates and per-minute buckets and then deletes the raw rows. Disk use is driven almost entirely by that raw window, which is why the window is short.

Logs do not survive that treatment. The value of a log line is the line. You cannot aggregate your way out of storing it, and the question an agent asks (show me what this app said around the time this job started failing, three weeks ago) is a question about rows you deliberately deleted. Scaling that shape up to every log record from dozens of apps, for a month, is the part our Postgres approach is worst at.

What we want instead is columnar storage that lives on the bucket. It should be cheap per byte, fine with being written in batches, and fast to scan when a query only needs one column.

Why not Quickwit

Quickwit was the obvious candidate, and somebody has already run it on Laravel Cloud and written it up carefully. It is a good write-up and worth reading before anyone repeats it. It covers a Rust wrapper supervising the Quickwit binary, the index and the metastore on an S3-style bucket, cache sizes tuned in the generated config, and warm counts in single-digit milliseconds.

Two things in it stopped us. The release it runs is Quickwit 0.8.2, from June 2024, and the author describes the project as being in maintenance mode under Datadog. The metastore also assumes exactly one writer. That is why the app is pinned to one replica, meaning one running copy of the app, with autoscaling off. And Cloud's deploys briefly overlap the old and new instance.

Neither is a reason nobody should use it. They are reasons we do not want to build a product on it.

We are trying DuckLake instead. DuckLake stores the data as plain Parquet files in object storage. It keeps the catalog in an ordinary SQL database, Postgres among them, with no catalog server of its own. On Laravel Cloud, that maps onto resources the platform already offers. The Parquet goes on the bucket attached to the environment, and the catalog goes in Cloud Postgres. When you attach a bucket, Cloud injects its credentials into the app, so there is nothing to configure by hand and nothing to supervise. DuckLake reached 1.0 in April 2026 and promises backward compatibility from there. That is a different position to be in than a release from June 2024.

Why this matters past logs

We want to optimize future versions of Hone, and Hone's storage is exactly the thing we just described as the wrong shape for high-volume raw data. If bucket-resident Parquet works here, it is a candidate there.

We would like a logging product, and this is the only honest way to find out whether we can build one.

And we keep wanting to build ingest-heavy things. A pattern for "accept a lot of small writes, store them cheaply, query them later without a warehouse" is worth having for its own sake, and we would rather learn it on something throwaway.

PHP now, Rust later

Ingest starts in PHP, because Cloud's Rust runtime is still in early access.

Once it is generally available, we would like a Rust ingest endpoint deployed as a second environment beside the original one. On Cloud, one application can have several environments, each deployed from its own branch. Connected apps would switch to it by changing the ingest URL they post to. Nothing else about the system would move. That would also show how versatile Cloud is. The same application would have its fast path rewritten in another language, running next to the original on the same platform.

For this experiment, that means two things have to be designed as if they were permanent, even though nothing else is. Those are the ingest contract and the on-bucket layout. Everything between them is the part we expect to throw away.

Open questions

These are the things we do not know. Nothing below has been measured.

  • Can PHP on Cloud drive DuckDB at all? There is a third-party FFI client. DuckDB lists it as a tertiary client, and it needs ext-ffi and a platform-matched C library pulled in at install time. The alternative is downloading the DuckDB CLI binary at build time and shelling out to it. That is roughly what the Quickwit write-up does with its binary. We do not know whether either works on Cloud's PHP runtime, or which architecture we would be downloading for.
  • Where does an accepted batch wait? Ingest should accept a batch and return, then write Parquet behind the request. But Cloud's gateway times a request out at around 20 seconds. Cloud's managed queues run queued jobs on separate compute rather than on the app instance. So "write it after the response" and "hand it to a worker" are not the same answer here. We are not sure either is available in the form we want.
  • Does compaction keep the small-files problem in check? Every insert writes a new Parquet file, and a log sink inserts constantly. DuckLake's answer is ducklake_merge_adjacent_files, which merges small files without expiring snapshots. Its documentation recommends files of at least a few megabytes. We need to measure whether a scheduled merge keeps up with forty apps' ingest, and what it costs in bucket operations.
  • Are agent queries fast enough? The realistic target is about forty apps and thirty days of retention. An agent asking a question is interactive. A query that takes a minute is a query nobody asks twice. We need to know what the common shapes cost over a bucket, not over a laptop.
  • Can the driver be invisible? A log driver that slows down a request, or throws when the sink is unreachable, is worse than no logging at all. It has to batch, give up quietly, and never become the reason an application is down.
  • What does all of this cost? Per app, per month, for a realistic volume, including the bucket operations compaction generates.

The risk we want to name now

Logs fed to an agent are a prompt-injection channel.

Anybody who can get a string into one of these applications' logs (a user agent header, a form field, a name, an error message quoting their input) can put text in front of the agent reading those logs later. That is not a hypothetical property of this design. It is the design. Centralizing the logs of forty apps and handing them to an MCP makes every one of those apps an input to the agent.

We do not have an answer yet, and we are not going to pretend the answer is "sanitize the input". Writing it down before we start is the minimum.

What we are actually deciding

This one is throwaway on purpose. The storage question is the one that decides it, and it is the one we cannot answer by reading.

If it works, we name it and rebuild it from what we learned. If it does not, we will know whether the reason was DuckLake, PHP, the platform, or the idea. Even then, the ingest contract and the bucket layout should still be worth keeping.

Experiments on this idea

  1. Experiment

    We built a central log sink on Laravel Cloud

    Ingest, compaction, two concurrent writers, the log driver, and the MCP all worked in the first hour and a quarter. Search did not. One cross-app search over 500,000 rows took fourteen seconds. The cause was 132 ranged GETs to fetch two megabytes. When we partitioned by day instead of by app and day, that query took 1.65 seconds.

    Read the experiment

Support the Lab

The Lab runs on our supporters

Every idea here gets tried for real: deployed, broken, fixed and written up. Right now that is one person doing the work between everything else, and supporters are what buy the time. As support grows, it will also pay freelancers to run experiments of their own, so more of these ideas get tested, sooner.

Our supporters

  • Chill Vibes

    A curated collection of fun and interesting experiments and projects built with AI to entertain and educate.

    Supporter Visit
    Visit Chill Vibes
  • Fawnfox Fables

    Create and publish your own children's books. Stories built around the people you love.

    Supporter Visit
    Visit Fawnfox Fables
  • Sputnik Intelligence

    Spain

    We transcribe the podcasts and read the newsletters, and make them searchable. Set up alerts, or hand it all to your AI agent over MCP or API.

    Supporter Visit
    Visit Sputnik Intelligence
  • Team of Me

    Free resources for indie makers and people thinking about striking out on their own.

    Supporter Visit
    Visit Team of Me
  • Wicked Viral

    Earth

    Connecting builders with promoters who can help them take their products to market. Get your product listed today!

    Supporter Visit
    Visit Wicked Viral