Idea
A central log sink for many Laravel apps
· Ed Grosvenor
Most of the organizations we work with run more than one Laravel application. Some run a lot of them. Their logs live wherever each app happens to write them. That might be a file on an instance that is gone after the next deploy, a hosted service one app was wired to years ago, or a channel nobody has looked at since it was configured. Asking a question that spans two of those apps means opening two things. Asking one that spans forty is not a question anybody asks.
So we want something like Papertrail for an organization with a lot of Laravel apps. Each app installs a log driver. The driver ships that app's log records to one ingest API. Every app's logs land in the same place, and the organization's agents explore all of them through a single MCP server. There is no per-app dashboard and no end-user accounts, just an admin UI and the MCP.
We know someone running about forty Laravel apps who would love one MCP that lets them talk to all of their logs. That is the whole product idea, and they are the reason we think it is worth an experiment rather than a conversation.
The experiment repository is artisan-build/ducklake-logs-experiment. It is temporary. We are finding out whether this can be built, not building it.
Raw by default
The driver sends log records unmodified. Whatever the application wrote, the sink receives.
That is a deliberate choice, and it has an obvious cost. Applications log things they should not, and once logs leave the app, there is one more place holding them. The alternative is to quietly drop fields and have somebody discover a year later that the one line they needed was filtered out before it was ever sent.
Redaction is a hook. The payload filter is a rebindable binding that the application overrides. An app that needs to strip a header, a token shape, or a customer field does it in its own code, where the people who know what is sensitive are. We would ship guidance written for an agent to follow. The app's own agent could then write that filter, instead of the person installing the driver reading a reference page and guessing.
Why not Postgres
Hone already collects telemetry from a team's Laravel apps and keeps it in the deployment's own Postgres database. It works because it does not keep much raw data. Individual events live for 72 hours by default. Hone rolls them into daily aggregates and per-minute buckets and then deletes the raw rows. Disk use is driven almost entirely by that raw window, which is why the window is short.
Logs do not survive that treatment. The value of a log line is the line. You cannot aggregate your way out of storing it, and the question an agent asks (show me what this app said around the time this job started failing, three weeks ago) is a question about rows you deliberately deleted. Scaling that shape up to every log record from dozens of apps, for a month, is the part our Postgres approach is worst at.
What we want instead is columnar storage that lives on the bucket. It should be cheap per byte, fine with being written in batches, and fast to scan when a query only needs one column.
Why not Quickwit
Quickwit was the obvious candidate, and somebody has already run it on Laravel Cloud and written it up carefully. It is a good write-up and worth reading before anyone repeats it. It covers a Rust wrapper supervising the Quickwit binary, the index and the metastore on an S3-style bucket, cache sizes tuned in the generated config, and warm counts in single-digit milliseconds.
Two things in it stopped us. The release it runs is Quickwit 0.8.2, from June 2024, and the author describes the project as being in maintenance mode under Datadog. The metastore also assumes exactly one writer. That is why the app is pinned to one replica, meaning one running copy of the app, with autoscaling off. And Cloud's deploys briefly overlap the old and new instance.
Neither is a reason nobody should use it. They are reasons we do not want to build a product on it.
We are trying DuckLake instead. DuckLake stores the data as plain Parquet files in object storage. It keeps the catalog in an ordinary SQL database, Postgres among them, with no catalog server of its own. On Laravel Cloud, that maps onto resources the platform already offers. The Parquet goes on the bucket attached to the environment, and the catalog goes in Cloud Postgres. When you attach a bucket, Cloud injects its credentials into the app, so there is nothing to configure by hand and nothing to supervise. DuckLake reached 1.0 in April 2026 and promises backward compatibility from there. That is a different position to be in than a release from June 2024.
Why this matters past logs
We want to optimize future versions of Hone, and Hone's storage is exactly the thing we just described as the wrong shape for high-volume raw data. If bucket-resident Parquet works here, it is a candidate there.
We would like a logging product, and this is the only honest way to find out whether we can build one.
And we keep wanting to build ingest-heavy things. A pattern for "accept a lot of small writes, store them cheaply, query them later without a warehouse" is worth having for its own sake, and we would rather learn it on something throwaway.
PHP now, Rust later
Ingest starts in PHP, because Cloud's Rust runtime is still in early access.
Once it is generally available, we would like a Rust ingest endpoint deployed as a second environment beside the original one. On Cloud, one application can have several environments, each deployed from its own branch. Connected apps would switch to it by changing the ingest URL they post to. Nothing else about the system would move. That would also show how versatile Cloud is. The same application would have its fast path rewritten in another language, running next to the original on the same platform.
For this experiment, that means two things have to be designed as if they were permanent, even though nothing else is. Those are the ingest contract and the on-bucket layout. Everything between them is the part we expect to throw away.
Open questions
These are the things we do not know. Nothing below has been measured.
- Can PHP on Cloud drive DuckDB at all? There is a third-party FFI client. DuckDB lists it as a tertiary client, and it needs
ext-ffiand a platform-matched C library pulled in at install time. The alternative is downloading the DuckDB CLI binary at build time and shelling out to it. That is roughly what the Quickwit write-up does with its binary. We do not know whether either works on Cloud's PHP runtime, or which architecture we would be downloading for. - Where does an accepted batch wait? Ingest should accept a batch and return, then write Parquet behind the request. But Cloud's gateway times a request out at around 20 seconds. Cloud's managed queues run queued jobs on separate compute rather than on the app instance. So "write it after the response" and "hand it to a worker" are not the same answer here. We are not sure either is available in the form we want.
- Does compaction keep the small-files problem in check? Every insert writes a new Parquet file, and a log sink inserts constantly. DuckLake's answer is
ducklake_merge_adjacent_files, which merges small files without expiring snapshots. Its documentation recommends files of at least a few megabytes. We need to measure whether a scheduled merge keeps up with forty apps' ingest, and what it costs in bucket operations. - Are agent queries fast enough? The realistic target is about forty apps and thirty days of retention. An agent asking a question is interactive. A query that takes a minute is a query nobody asks twice. We need to know what the common shapes cost over a bucket, not over a laptop.
- Can the driver be invisible? A log driver that slows down a request, or throws when the sink is unreachable, is worse than no logging at all. It has to batch, give up quietly, and never become the reason an application is down.
- What does all of this cost? Per app, per month, for a realistic volume, including the bucket operations compaction generates.
The risk we want to name now
Logs fed to an agent are a prompt-injection channel.
Anybody who can get a string into one of these applications' logs (a user agent header, a form field, a name, an error message quoting their input) can put text in front of the agent reading those logs later. That is not a hypothetical property of this design. It is the design. Centralizing the logs of forty apps and handing them to an MCP makes every one of those apps an input to the agent.
We do not have an answer yet, and we are not going to pretend the answer is "sanitize the input". Writing it down before we start is the minimum.
What we are actually deciding
This one is throwaway on purpose. The storage question is the one that decides it, and it is the one we cannot answer by reading.
If it works, we name it and rebuild it from what we learned. If it does not, we will know whether the reason was DuckLake, PHP, the platform, or the idea. Even then, the ingest contract and the bucket layout should still be worth keeping.