Was this taken from the Crowdstrike repo?
Was there even tests?
Yup, and they’re run on an estimated 8.5 million test machines
Testing in prod is a power move honestly. Rock star-level
Yeah, but I think apparently the tests that “could” have caught it relied on mocks which basically rendered it useless in those cases.
Ah yes. The unintended consequences of mandated code coverage without reviewing the tests. If you can mock the shit out of the test conditions to always give you exactly the answer you want, what’s the point of the test?
It’s like being allowed to write your own final exam, and all you need to pass the exam is 90% correct on the questions you wrote for yourself.
Unit tests, yes, but you don’t only do unit tests. Integration and e2e tests still exist.
And this is why mocks are bad…
Who needs tests when you have users?
The testing environment is production!
I work in state government and we insist on building our own systems. This is too fucking true and why it’s funny.
What do you mean building our own systems?
We build systems to manage day care licenses in the state instead of buying someone else’s product.
-Crowdstrike
We work in protoduction.
Technically there’s no substitute for testing in production lol
Although ideally you’d want to test it beforehand…
There are so many more, and better!, options than testing in prod, but they take time, money, and talent and ain’t no company got time for that (for a business segment that “doesn’t generate revenue”)
Been microsofts strategy for how long?
Probably too long. That was a philosophy I had at school and iirc the founders never finished school and started MS in a garage.
When the CI takes longer than 10 minutes
Envious. It averages 15-30 mins
Are you like building a mobile app or have 100k tests or is it just super slow?
Still waiting on approval for more resources. It’s not a priority in the company.
I swear we have like 4 runners on a raspberry pi.
My test suite takes quite a bit of time, not because the code base is huge, but because it consists of a variety of mathematical models that should work under a range of conditions.
This makes it very quick to write a test that’s basically “check that every pair of models gives the same output for the same conditions” or “check that re-ordering the inputs in a certain way does not change the output”.
If you have 10 models, with three inputs that can be ordered 6 ways, you now suddenly have 60 tests that take maybe 2-3 sec each.
Scaling up: It becomes very easy to write automated testing for a lot of stuff, so even if each individual test is relatively quick, they suddenly take 10-15 min to run total.
The test suite now is ≈2000 unit/integration tests, and I have experienced uncovering an obscure bug because a single one of them failed.
I used to have to use a CI pipeline at work that had over 40 jobs and 8 stages for checking some sql syntax and formatting and also a custom python ETL library that utilized pandas and constantly got OOM errors.
They didn’t write any unit tests because “we can just do that in the CI pipeline” and if you didn’t constantly pull breaking changes into your branch you would guarantee the pipeline would fail, but if you were lucky you only had to restart 30% of your jobs.
It was the most awful thing and killed developer productivity to the point people were leaving the team because it sucks to spend 40% of your time waiting for CI scripts to fail while you are being yelled at to deliver faster.
Who needs integration testing when we have users who will do it for us?!
Lol. I run an open-saas ecom and everything is done live. No one but me handling it. The customers must think they are tripping sometimes. Updates are rarely perfect the first push.
FUCK IT!!! WE’LL DO IT LIVE!!!
“YOLO” - more than 1 merge comment I’ve seen.
“Works on my machine”
Mo’ unit tests, mo’ problems.
This is my favorite commit comment of all time.
it’s not a commit comment
I don’t actually know enough to know anything about this but I’m assuming that’s badass and you can only do it with sunglasses on
Badass or stupid and incompetent? Take your pick lol
If QA cared about us they wouldn’t make it so hard.
I’m about to do this to this kernel driver. Certainly broken before, possibly broken after, what’s the worst that could happen
ya man fuck those losers
Also me at work.
