Threads
Loading user profile...
That's a whole lotta —'s in that there report chief
deterministic?! but sir it's 2026 and coding is solved! /s ... good luck! you've figured other things, you'll figure this one out too
That's a nightmare, I wonder if you would have luck combining something like [https://github.com/vercel-labs/opensrc](https://github.com/vercel-labs/opensrc) \+ pointing it to [https://github.com/theory/pgtap/](https://github.com/theory/pgtap/) locally. A readonly version of the supabase MCP could be useful to query pg? I haven't explored the read only capabilities of sb's MCP so be careful what urls/envs you allow the agents to point at.
Fair enough. I would push back a little bit, or rather keep you honest on the first point. Have you performed thorough artillery test comparison of this, or is it just a spidey sense / hunch? Not implying it's wrong, but just curious if you are going by feel here or a measured approach which showed the added overhead was absolutely unacceptable / risk to your performance. It sounds like you are very married to the concept of db-driven business logic enforcement. i would be hand rolling a unit testing framework for this as a serious asset to your business. With today's model harnesses this shouldn't be 'too difficult' to achieve.
Do you genuinely need stores procedures and triggers? Is there anyway you can shift that out of the db layer and orchestrate the event nature in any other way?
Do not do this without explicitly creating rules to never edit your schemas directly or you will soon have no idea what's happened after random tasks. Learn git. Work on small tasks one at a time. Check out aihero.dev
That seems sensible to me, you might get away with downloading the whole csv in memory and have 1 async function per user upload. Hold on to original. Work on the file and upload that to a separate storage once it is processed. Hold on to the original files (see last thought) Honestly what you have seems fine to me, it's probably more a case of fine tuning your context window on the model you're using, playing around with results until you find the most efficient model for this. You may want to look at NMTs or TLLM Vs your run of the mill generic models. To answer your question about boundaries, I would consider carefully the nature of the data and whether you are allowed to process it beyond translation. Store user data purely as upload/dump. If you can afford it, I would periodically upload each row or samples of these to postgresql to be able to run agentic metrics on the kinds of data you are translating at scale. Future you will thank you later when you understand better how to retrain models suited to whatever domain your saas is targeting. The data is the most precious asset you have, treat it as such.
What in the brute force llm abuse is going on here. I fear for the future of software - please tell me you have a deterministic data cleanup/processing pipeline in front of whatever 'enrichment' you are doing here. Would probably go towards temporal or trigger.dev or step functions to orchestrate the processing loop. What is running the async workflows now? What kind of volume do you have? You are right in leaning away from in memory processing, would be good to know what this product is trying to solve for to have a better idea of what you need architecturally
All fun and games until you get a network glitch, email service has a 500 or the edge is temporarily unavailable. If you're gonna go this at least implement an outbox pattern to not couple your functionality to a hidden brittle trigger. Fire and forget is fine if you have a recovery mechanism, this ain't it I'm afraid.
If I was worried about backup I don't think I would be on a free tier. And if I was worried about backup on a free tier I would be looking at hand rolling a backup solution like you are suggesting. I would still encourage you to build what you are building to practise SaaS modelling, but I'm afraid I think you're targeting the wrong group. Someone on a free tier is likely on a free tier for hobby reasons.