Technology
Modern JavaScript Testing: Integration, Contract Testing & AI Tools - JSJ 692
In this episode of JavaScript Jabber, hosts Steve Edwards and Dan Shapir are joined by guest Yanni Goldberg to explore the evolving landscape of JavaScript testing. They delve into the latest trends, ...
Modern JavaScript Testing: Integration, Contract Testing & AI Tools - JSJ 692
Technology •
0:00 / 0:00
Interactive Transcript
spk_0
Hello everybody. Welcome to another thrilling episode of JavaScript Jabber. I am Steve Edwards,
spk_0
the host with the face of radio and the voice for being a mind but I'm still your host.
spk_0
With me on the panel today all the way from Tel Aviv, Israel's Mr. Dan Shapir, how
spk_0
you doing Dan? I'm doing well. How are you Steve? Doing good. Doing good. It's Wednesday
spk_0
morning, first day back at school for kids here so that's always good. Still hot and
spk_0
toasty there in Tel Aviv as always. Yeah. When is it not? Yeah it's a little cooler here.
spk_0
Starting to cool down a little bit which is always a bummer but it is what it is. And our guest
spk_0
today coming all the way also from Israel, Mr. Yanni Goldberg, how you doing Yanni? Hey hey, great,
spk_0
great time. Really happy to be here today. It's one degree cooler here comparing with Tel Aviv.
spk_0
Very hot. But the good news that after two months of being at home kids are finally back.
spk_0
It's cool yesterday or before so yeah back to work a little. Yes and for those of you that might
spk_0
see Yanni drink, he is not drinking and podcasting that is tea not beer. I clarified that before the
spk_0
episode so not that it's not okay to drink and podcast but just terrifying. I literally drank beer on
spk_0
stage while giving certain talks. My contention is that it only made the talks better. Sorry,
spk_0
we'll listen to you up a little bit. Is that it? Yeah. Did you probably know is it also legitimizing
spk_0
the states? I know it's you in Europe, it's okay but if I drink at like at the states during
spk_0
a talk would it be considered PC? I've never heard anything against it. I guess it depends on how
spk_0
well you handle the the alcohol and just stay on topic and have somewhat of a sense of humor than
spk_0
your saying that the first two are okay. In Germany it's perfectly legit to have beer at lunch
spk_0
during work days and is it legit or required? Yes, there is something to that. In Paris and it's
spk_0
really good beer. When I worked at Wix, let's just say that we worked hard and played hard and
spk_0
sometimes played hard during working hours. Alrighty. So with that we will transition into the topic
spk_0
du jour, the topic of the day which is JavaScript testing and what's going on in testing in the
spk_0
JavaScript world. So you only take it away. Where do we want to start? I'm sure you wanted to start
spk_0
by sharing a few words about myself and my... No, we don't care about that. We just want to know what
spk_0
you have to say. No, I just... Yes, seriously, sorry, I'm a little off track. Tell us who you are,
spk_0
why you're famous and where people can give you money or we'll say that for the end. Just tell us who
spk_0
you are. Sure thing. So, surprise, surprise, I'm a developer freelancer in a consultant doing
spk_0
full stack, a lot of JavaScript, but more than anything else, my really my love, my passion, the
spk_0
things that I'm trying to specialize in is that all developer workflow and specifically testing.
spk_0
I think that I've worked with more than, you know, 40 organizations, some of them are big
spk_0
now you're probably recognized on improving their testing. So I literally said over the shoulders
spk_0
of like hundreds of developers as we try to craft tests that are both efficient and yet simple,
spk_0
done a lot of mistakes, learned a lot, tried a lot of new frameworks and I hope to share some of
spk_0
these lessons here today. Isn't the process to your basically cursor right test for me?
spk_0
Exactly, exactly this. Actually, where I'm about to cover, I would also cover some kind of
spk_0
testing AI trends, mostly the useful parts of AI. So yeah. Okay, just to finish that particular point,
spk_0
the scary thing is that afterwards when you make changes and then the test break, you tell us,
spk_0
cursor fix the test for me and you're not sure whether it's fixing your test to hide the bugs
spk_0
or fixing your code to fix the tests or doing both or whatever, it can be really interesting.
spk_0
That's pretty good. Definitely. I mean, I think that one of the first thing one should tune in
spk_0
cursor or cloud code or whatever you're working is that you can let it work in yellow mode,
spk_0
but I think there is one exception. Any human confirmation should be made any time a test is
spk_0
changed. You may change my production code. I may inspect each and every nine or not, I don't know,
spk_0
but a test change should supervise that that's my that's a good take, I think. I can't take it
spk_0
anytime. I'm just going to say here write me some code and then commit it or do whatever without
spk_0
looking over it and making sure it works. Well, yeah, that's easy to say and everybody starts with
spk_0
good intentions, but at a certain point in time, you know, especially when you're under the gun
spk_0
and certain things and things seem to work and you build a certain level of trust, I've known
spk_0
people to almost even engineers almost get to the point of almost vibe coding their ways through
spk_0
projects. And I think Yoni's statement that if you've got good test coverage, then ensuring that
spk_0
the tests are properly maintained and reviewed is a good way to add a significant layer of trust
spk_0
and security around your vibe coding. Yeah, I agree with both. I think that
spk_0
people have various degrees of how much they supervise AI generated code.
spk_0
I'm just saying that the least conservative one can be is at least guard your testing. You might
spk_0
do this also for the production code, but I think that letting AI also change test as it change
spk_0
code. This is the highest highest amount of risk one may take. Yes, okay then. So what are the
spk_0
latest and greatest trends when it comes to tests aside from having AI do all the coding for us?
spk_0
Yeah, so I'll kind of mention some kind of super trends and I very impactful changes and then we can
spk_0
dig the dive deeper into each one of them. At least the ones that we found interesting.
spk_0
So I think there is always in testing one kind of everlasting progress in which we always every year,
spk_0
every we try to get our test to be more production like without paying the price of so in the
spk_0
past two years, we keep seeing this progress and in front and the front and the world for the first
spk_0
time I think like we had react testing library ruling this world for a lot of time,
spk_0
test the trance in OJS. I think that's for the first time it's safe to say that tests that don't
spk_0
run in the browser that are not visual are almost out of scope. There are many new ways and frameworks
spk_0
to run your test in the browser and get the full visual experience and the confidence. So it's
spk_0
more like production but some of them run really fast. So you're not paying the price of like
spk_0
full a full blown end to end framework. On the back-end side it means that you're having a
spk_0
microservice you can more easily now run it on your computer with all the layers and almost
spk_0
like production infrastructure and again without paying tremendous price. I think these are bold
spk_0
and changes that we're seeing and we I guess we will dig deeper into the specific tooling
spk_0
and just to clarify there are obviously layers to tests. So they're I don't know if you know you
spk_0
should refer to them as higher or lower layers it's just different positions in the testing cycle.
spk_0
So you've got obviously stuff like unit tests which should be very relatively small,
spk_0
relatively straightforward and certainly very very fast and they usually don't need special
spk_0
wiring although you know certain degree or certain amount of mocking is often involved.
spk_0
Then you've got integration tests and you've got end-to-end tests. It seemed to me that when you
spk_0
were talking about the improvements in our ability to do effective efficient and performant
spk_0
testing you were mostly talking about integration tests and end-to-end tests and less
spk_0
about unit tests correct? Yes and a little no. I mean I think that first there is a trend. I'm
spk_0
I'm not advocating by the way on anything I'm just trying to cover what like trends and that
spk_0
as I observe and I think another trend is seeing less unit tests and more kind of component page
spk_0
integration tests or to call this the testing diamond it's a kind of an emerging concept where
spk_0
you should write more integration tests but with the new tooling even if you write unit tests
spk_0
like take a framework that we will cover that is called Vita's browser mode even your unit
spk_0
tests run in the browser. Now think about it not all the unit tests in the world are pure. You
spk_0
might test the modules that actually doing some stuff with the web APIs so instead of running it
spk_0
in no JS with polyfields and trust and trust the similarity it runs your test in a real browser
spk_0
in chromium for example so even your unit test gains more kind of a production like experience.
spk_0
Does this make sense? Yes it does although I need to think about what I think about it.
spk_0
It's kind of a recursive statement because really when I'm thinking about unit tests I'm really
spk_0
trying in most cases to be as stateless as possible and consequently not be dependent on having access
spk_0
to a browser dome which is kind of like persistent by definition and so I'm not sure
spk_0
that this is where I want my unit tests to go necessarily but I certainly appreciate the fact
spk_0
that we don't need to mock browser APIs, dome APIs which is always let's say less than ideal.
spk_0
But why do you think that is? What has changed that has made this? What has changed that has
spk_0
made this possible now that was not available before? Yeah first of course,
spk_0
I'm just mentioning that I fully agree the real value is in integration tests,
spk_0
not about page testing or component tests also in end to end in unit tests it's indeed less
spk_0
significant. So it's mostly if we if we if we double double click into the front end word for
spk_0
a start I think now there is a key up from decisions that teams should make there is kind of a
spk_0
triangle decision between three different frameworks for testing components and pages and one of them
spk_0
is relatively one of them is is new it's worked the past two years and it makes running visual
spk_0
testing the browser almost as fast as unit tests. So there was a lot of people trying to avoid that
spk_0
kind of visual experience because of the hassle involved but with some of the new frameworks like
spk_0
VTest browser mode and the new storybook model it becomes easier faster so you have less reasons
spk_0
not to just hey let's test it like where the user lives and maybe at some point explaining
spk_0
what is VTest browser mode how does it stack with playwright go for it now is that the same as VTest
spk_0
UI is browser mode not exactly because you can we can also have VTest UI with unit tests
spk_0
and browser mode is a new sub framework sort of a test and to zoom out for a second I think that
spk_0
we have now three measure and good options in the front end word that one must make a decision
spk_0
the first one is playwright the good all-known playwright a very reputable
spk_0
very reputable framework that is being very popular which allows you to test your system essays
spk_0
like you can spin up a whole page and you get a real page in a browser but you get all the
spk_0
you get the entire you get the entire application with the router and the state and a lot of
spk_0
things beyond the page level or the component what you want to test so the performance is like
spk_0
a little of concerning playwright even when you walk the back correct me if I'm wrong a playwright
spk_0
is essentially just a hellish chrome with APIs to automate various tasks that's more or less
spk_0
right yes and maybe a little more because it can run your test in other browsers as well like run
spk_0
the test once and get it running in chrome or Safari or anything yeah because they basically later
spk_0
on added support for the same APIs is is I recall exactly they ported kind of a CDP protocol for
spk_0
other browsers exactly but on top of this they also have a very nice and stable API to make
spk_0
to make the experience of interacting with pages less flaky so for example if you're trying to
spk_0
ascertain something they will take care to retry and wait and until that condition is met on the
spk_0
page so the overall testing experience is really is really great and and it replaced all the
spk_0
frameworks like Cypress and Selenium right exactly exactly this okay so how is VTES
spk_0
browser more different from playwright yeah so first worst mentioning mentioning that VTES
spk_0
browser mode is using playwright under the hood so you get a bottle tested engine that you can
spk_0
trust but what they try to do playwright is a little bit disruptive to front and testing so if you
spk_0
take traditional front and testing with people used to to work with things like react testing
spk_0
library the syntax of reality testing library the concept of react testing library supporting from
spk_0
there to playwright was a little disruptive VTES browser mode allows you to take your existing test
spk_0
or learn nothing new use the same syntax same concept only instead of running it in no JS like
spk_0
react testing library you run it in real browser which obviously allows you to see you probably
spk_0
know that that's really nasty experience when you test with react testing library and a test fail
spk_0
what you'll see on the screen is 300 lines of XML of HTML in a CLI in which you try to understand
spk_0
in this my component how does it look like what's wrong here I mean nobody wants to
spk_0
travel should test in a HTML over CLI with this browser mode you can see them in a browser
spk_0
and benefits from all of the chrome developer toolbar perks
spk_0
so that's that's the premise that's the obvious upside they also make it's like you know VTES
spk_0
stands for fast in French VIT the word VIT is fast in French and they they stand up to their
spk_0
reputation like I can I already use it in production it's amazing I can like test in
spk_0
tire page some interaction some kind specifics and now you're in 200 milliseconds it's like blazing
spk_0
fast you click on a button and you you you don't have enough chance to see what happened on the
spk_0
screen so the performance is really awesome with that there are some concepts should be
spk_0
mentioned as well it's very new it's still tagged as experimental for like two years or so
spk_0
and so it's it's it didn't yet stand the battle of time like playwright
spk_0
and there are many other other decisions that were made that are that might appeal to some teams
spk_0
and not to to others but it's definitely a bold opportunity I think now in 2025 for testing and
spk_0
I could not finish that I'm sorry this is kind of a mini mini TED talk but there is also storybook
spk_0
and storybook now change their model and now they're using VITES browser mode under the hood
spk_0
which use playwright so you have three options you know that Russian doll which is called
spk_0
I believe matroyska babushka or matroyska yeah it's kind of a nested doll
spk_0
yes so what we have now in the in the testing space it's it's it's it's a matroyska so you have
spk_0
playwright or VITES browser mode it's use playwright or storybooks that use you got the idea and you
spk_0
have to make you have to make the choice so if you're starting a new front end project today and
spk_0
let's say that you're using one of the two leading options I don't know how familiar you are with
spk_0
them either react which is approximately 50% of the market or view which is approximately 20 to
spk_0
30% of the market and you need to do and you want to do testing because we we started with
spk_0
testing for the front end what would you use yeah that's that's first a very I think that the
spk_0
choice is really hard today I had to choose to choose for a customer like two weeks ago and then
spk_0
the decision is really hard so from one hand I lean towered being conservative and say hey let's
spk_0
use playwright it works it's like the most simple it's like the the last layer it's simple it's
spk_0
proven let's just use it again playwright in this case is for mostly for the end-to-end tests right
spk_0
or you can use it also for component for component or page testing it it has great tooling to
spk_0
allow you to mock your backend and I think I think that for for the majority of the testing
spk_0
front end team want to mock the backend but yeah you can also run in well let's put it this way
spk_0
if you're not mocking your backend then you're by definition dependent it's almost it kind of
spk_0
almost means that by definition your tests are going to be flaky at least some of the time if you
spk_0
do involve the backend if you do involve the backend so so yes if you want to go for separations of
spk_0
concerns you want to mock external services reduce the amount of dependencies that you have on
spk_0
external services and that includes your own backend totally agree it's slower it's more flaky but
spk_0
my my my highest concern with end-to-end is actually that it's really hard to simulate things
spk_0
beyond heavy passes so think about it you want to test now that the user is disabled it's a
spk_0
premium user you get 20 records all of the things that are beyond the simplest response
spk_0
with real backend it might be very hard to tune the backend into each and every scenario that you
spk_0
want however when you mock it you can simply just say hey I want this response I want that
spk_0
response yeah on the other hand when you're mocking it you're basically inventing the responses so
spk_0
it's not necessarily how the system behaves it's not really end-to-end if it's if it's mocking
spk_0
I mean wouldn't you normally like spin up like a and maybe this is what you're talking about with
spk_0
mocking but basically create a test database or a copy of your database spin one up seed it you know
spk_0
you got your data in it and then run your test actually using a database it's not your
spk_0
prod database obviously but at least you have a clone in a real database that you're interacting with
spk_0
because I know you can personal experience a lot of times that's that's valuable testing
spk_0
yeah yes but all I think that first I'd be before answering that I'd like to repeat what dance
spk_0
and I think it's it's like a key point here if you're doing is mocking okay I'm just walking
spk_0
some Jason yeah you are now a tourist of doing things that are not exactly the same in production
spk_0
and this is why you also need contract testing and I think this is definitely a topic we want to
spk_0
double click into later because there are new interesting tooling in the JavaScript space related
spk_0
exactly with this problem how do you create smart and aligned mocks now to this one yeah in a simple
spk_0
case it's just crowd applications so you set up something in the database and you and you
spk_0
display them but in many cases creating the server response is much harder than this let's
spk_0
to give you an example one of my customer a user entity and it might get disabled how does it
spk_0
get disabled there is some cron job running every minute and concluding that the user is disabled
spk_0
so how would you create a disabled user from a test and make the back end run a cron job
spk_0
and tag it it's it becomes much so much harder than just returning one line hey just give me a
spk_0
disabled user Jason am I being clear on this point no it's it's obviously true I like getting
spk_0
an application to do what you exactly what you want and it can be challenging and and configuring
spk_0
a certain back end application to replicate certain scenarios is a problem just think about
spk_0
you know the scenario where a customer tells you to they are experiencing a certain problem and
spk_0
maybe you even have traces that kind of show the problem the way that the customer is experiencing
spk_0
is and now you're trying to replicate that scenario in the lab in order to debug it we all know how
spk_0
challenging that can be so I totally agree that replicating weird behaviors or edge certain edge
spk_0
behaviors that are legitimate and you want to test for in an in actual environment that simulates
spk_0
production can be challenging and it's often much easier to just play recordings of back end
spk_0
responses and and you know effectively mocking the back end it can be much much easier
spk_0
likewise also mocking the front end you know you you want to test the back end system then you
spk_0
might use some sort of I don't know just post things out of postman or whatever or playwright
spk_0
rather than actually having an actual front end that you know you create a process that clicks
spk_0
through it can be much more challenging and more complicated and lengthier
spk_0
so is there a point you know yawning with that example you gave with the cron job
spk_0
is there a point where you have to say okay we can't test everything in an automated test and
spk_0
maybe we have to do something manually or do you always work under the assumption that everything
spk_0
in product can be handled with an automated test yeah so I'd say based on my experience as long
spk_0
as you stick to gray box testing when I say gray box testing it's a test that you don't test
spk_0
the entire system you test one component you test like the user like you just put things in
spk_0
and expect some outcomes but you also run on the same process like that application you can simulate
spk_0
anything I didn't encounter anything that is not simulated that it's hard to simulate as long as
spk_0
you are your test and the and the co-ed under test be within the same process I definitely in terms
spk_0
of ROI yeah definitely prioritize and focus on first on the things that the user does and the things
spk_0
are likely to happen and where the bugs are more severe but in terms of the technical capability
spk_0
of simulating stuff I think that today's in the JavaScript ecosystem I rarely encounter the
spk_0
situation that something was not reproducible and I'll add to that that manual testing is kind of
spk_0
by definition anecdotal and I hate to depend on anecdotal evidence of correctness
spk_0
and so even if you do certain amount of manual testing then record that testing and then turn
spk_0
it into an automated script it doesn't need to stay manual forever so a certain amount of manual
spk_0
testing especially for I don't know complex systems might be required but again try to automate
spk_0
it going down the line definitely agree I also have if someone is interested in articles that's
spk_0
called testing the dark scenario of your backend in which I've shown in an OJS application how in
spk_0
each in in seven lines test not no test there was more than seven lines you can test really critical
spk_0
scenarios it usually teams don't test like for example test that when your bootstrap your your
spk_0
backend bootstrap phase fails then you then you get the right monitoring or test that when your
spk_0
live liveliness route is not behaving correctly you get the right thing or when you know very common
spk_0
in JavaScript word when there is a process unquote exception or a handed rejection is happening
spk_0
in your application do you do the right thing this is a critical event I mean your process might
spk_0
just crash do you test this I showed out like in a five lines of code you can also cover this
spk_0
advanced error handling facets so yeah things it's so again to summarize what we've talked about
spk_0
so far so basically what you're saying is that you're kind of promoting more sophisticated tests we
spk_0
usually component level tests or integration tests less unit tests and that using modern tooling
spk_0
that you mentioned like playwright like vtest browser mode and like
spk_0
storybook testing it's much easier to do than it used to be because basically what you're saying
spk_0
if I am the same correct yeah much easier than before and I'd say I mean I'm not against unit
spk_0
tests but I think that for a start your first step should be what is the user is doing and the user
spk_0
is not stretching functions or models right the user is visiting a page so I think that the first
spk_0
things that you should test is your page how the user interrupt and then as you call if you
spk_0
realize some complex highly complex logic that you want to isolate testing isolation for better
spk_0
whatever performance then yeah go for unit tests but I think this is kind of
spk_0
context to all and optional well to be fair unit tests actually were kind of blown out of TDD
spk_0
so initially the concept of unit tests at least for people who take that approach is basically
spk_0
we first write the tests then we write the code now AI writes the tests and the code so
spk_0
that's less that approach to be also to be fair I've never been able to adopt the TDD model maybe
spk_0
I'm too much set in my ways it just doesn't seem to be the way that I think but I have
spk_0
gotten value from unit tests the main advantage of unit tests is they're they're very lightweight so
spk_0
they're almost effectively free to run and they run and I have them run constantly because I you
spk_0
know I don't care and if something breaks it's there they test very specific things so you get an
spk_0
instantaneous indication of where that problem is so you don't need to think about like with then
spk_0
let's say especially with end-to-end tests you know the test fails okay why did it fail it often
spk_0
requires pretty significant amount of investigation in order to understand why an end-to-end
spk_0
test failed and like we talked about before it could also be related to certain things that
spk_0
do you know flaky tests because you're dependent on external services with unit tests you're not
spk_0
dependent on everything and anything you're just testing your own code it runs very quickly you're
spk_0
testing a very specific function let's say and you know I've introduced I made a change in that
spk_0
function that's why it breaks couldn't agree more you know I observed once something very
spk_0
interesting that can't be back for those of you are not familiar with can't be is kind of the
spk_0
father of DDD a person who should really appreciate unit tests and he said in one of his blog posts
spk_0
unit tests give up predicting suitability to production and inspiring confidence so it's not
spk_0
about product confidence at all it's about being writable fast and specific now I think that most
spk_0
of the developers are looking in testing for confidence and yes it turns out it's not only about
spk_0
this yeah there's that famous picture about the difference between unit tests and integration tests
spk_0
about you know building a new renal in such a position that when you open the bathroom stall door
spk_0
you can't actually get at it so everything is fine at the unit level at the unit level but when
spk_0
you put it all together it doesn't work the only we've seemed to have lost your picture you
spk_0
still with us yeah I'm working on fixing the camera the good news is that I'm still online yeah okay
spk_0
okay and I think it was just revisiting for one last minute that that crucial decision
spk_0
between the three frontend things so we've mentioned playwright I think that storybook and
spk_0
Vitas browser mode are also great options and we didn't cover storybook so storybook if you
spk_0
have a design you ask me what would I pick so if we're having a design system and rely a lot of
spk_0
small components that we tend to communicate and test in isolation storybooks gives a very compelling
spk_0
package in which we get first a great workflow of visualizing our component and documenting them
spk_0
but also a great new testing experience so you write a test once and you get both accessibility
spk_0
testing if you have some accessibility issue you are covered visual regression testing without
spk_0
writing any other test out of out of the box if you're if you are out of the box and if you
spk_0
are sharing your credit card visual testing and also also functional testing using playwright
spk_0
under the hood so you get when you say visual testing just to verify that I'm you understanding
spk_0
your terminology you basically mean that if I accidentally I don't know change the font or
spk_0
something it will break my test because the content the actual pixels on the screen are not the
spk_0
same as they're supposed to be exactly it should be smarter than this and it should realize and detect
spk_0
significant changes from like minor improvement but exactly this since that functional test can
spk_0
tell you about so your side menu is screwed something something I don't know something is
spk_0
totally broken on the screen a big change in the layout you put a red background or something
spk_0
for testing purposes and then forgot to remove it so that's that's that's a nice example yeah
spk_0
so storybook gives you all this together an interesting option on the other end it's the
spk_0
more sophisticated option it's it's all the storybook narratives and with the with us browser
spk_0
mode on the wood hand playwright so you have a lot of moving pieces to handle is it open source
spk_0
is it a paid product or a service what what is it it's open source it's a free product except I
spk_0
think this is how they make the revenues except the visual regression test what you just mentioned
spk_0
this one is a paid service okay cool hey just to clarify terms I know we've talked about different
spk_0
types of testing we talked about unit testing integration testing and then testing visual testing
spk_0
the one I always get hazy on is a difference between I guess what integration testing is how
spk_0
you find that versus unit tests versus and as test what and excuse me end to end testing so what's
spk_0
what's your definition of integration testing yeah without understanding that it's hard to get
spk_0
dance toilet joke about integration testing so yeah testing thermon terminology is totally broken
spk_0
I think that if when you say end to end testing or or even unit testing and and integration
spk_0
testing made different people will understand different things and for this reason Maya
spk_0
suggested in every company in your company name a very specific name for your test which is
spk_0
not integration and end to end take for example just one example in one of my customer we used to
spk_0
test pages we take a front end page we isolate it and the test cop is every time one page we call
spk_0
it page testing and everyone everyone understand what page testing mean it's a test of a page
spk_0
integration test just mean might mean three different things to different people
spk_0
if we try somehow to clarify the integration test is I think the most common definition of it
spk_0
that we test the user journey in a sync with test the whole sync user journey in a single
spk_0
tier of the system not end the entire system so it might be the user visiting some front end but
spk_0
without stretching the entire system it might be a user visit a call coding some back-end API
spk_0
and expecting your response this is our examples for integration tests so correct me if I'm wrong
spk_0
it also includes for example testing components in isolation so let's say I implemented a date
spk_0
picker component that date range picking component because you know you don't get that out of
spk_0
the box in the browser and I want to verify that it's working correctly and I can host it within
spk_0
like independently and basically simulate interactions with it it's not end to end because it's not
spk_0
done in the context of the entire application so it's not for example if you think about I don't
spk_0
know let's say some sort of a hotel booking application it's not done in the context of actually
spk_0
booking a stay at hotel it's done in isolation but it's still done at the level of actual user interface
spk_0
and interactions not just invoking functions and checking return values. Exactly you're in other words
spk_0
you are testing the user experience with this in its almost real environment there is a browser here
spk_0
there is your code there is a like you have multiple layers here orchestrating together you have that
spk_0
UI kit usually the UI kits that you're using and then you have your code the trapping it
spk_0
and you have usually sometimes your state manager and there is the browser all the moving pieces
spk_0
that exist in production are playing together in this test in this regard it's integration
spk_0
and if you're talking about non-frontend stuff for example a node system it might be testing a
spk_0
particular specific API call again not in the context of an actual complete user flow and potentially
spk_0
even with certain other parts of the system being mocked like we said before and not actually
spk_0
working with let's say you're working with various external services you're not actually working
spk_0
with these services you're you're mocking them and you're just testing effectively your particular
spk_0
let's say microservice or particular aspect of a microservice in isolation.
spk_0
Yes exactly what what I observe that works great for backend teams is testing your entire microservice
spk_0
essays put the infrastructure and local Docker compose test all the layer of the microservice
spk_0
if you deploy you test it your data access layer your business logic your API
spk_0
but only intercept calls to other services otherwise it becomes end-to-end with all the
spk_0
downsides that we have mentioned. Yeah I'm currently actually doing exactly that and an
spk_0
project that I'm working on being able to use Docker compose to effectively
spk_0
recreate the entire system on your own particular on your local developer machine and then
spk_0
be able to dynamically deploy changes to the specific microservice that you're working on
spk_0
is is like is effectively a godsend it makes life so much easier when developing in such complex
spk_0
environments. Definitely and it's surprising how fast this test is like I think usually
spk_0
like a 50 approximately 50 tests for microservice on a developer machine with real database
spk_0
less that last less than 20 seconds. Oh yeah it's a really efficient technique and maybe
spk_0
it's worth mentioning if we already that discuss backend that there is now a built-in test
spk_0
trunners in Node.js mostly for backend I think and it's really gaining a lot of momentum.
spk_0
Yeah it's it's interesting that it seems that between node implementing its own testing built-in
spk_0
testing capabilities on the one hand and VTest on the other they're kind of cornering the
spk_0
testing market in a sense and I recall for example seeing this post on X I think from
spk_0
a KENSI DODES talking about how happy is that the React testing library is no longer needed because
spk_0
effectively all this functionality is now built into VTest I think. Yes and I think it's really
spk_0
interesting to see how people are contributing libraries and ideas together so it's the
spk_0
React testing library was a library but it was also a philosophy of how you test. Now the library
spk_0
probably fading away or at least losing some momentum but the concepts that the concepts there
spk_0
that the techniques are being totally used in VTest browser mode and in playwright and so many
spk_0
new testing frameworks are taking inspiration and building on this that's it's very very
spk_0
interesting to see this now evolving process. So in general what would you say is it's still
spk_0
much more difficult to test the front end than the backend or are we achieving some sort of parity?
spk_0
Yeah I tell you that I'd easily say that testing the backend is much much easier than testing the
spk_0
front end. Yeah I mean first you can also judge it by the tooling I mean there are
spk_0
accepted new test runners of Node.js that is actually bring less features it's funny because the
spk_0
new test runners that a lot of backend teams are now using it's brings much less testing features
spk_0
and what used to be like ingest set. People don't need even all the features and there are very
spk_0
few like new tooling in the backend except maybe contract testing tooling which we will mention.
spk_0
However you see that in the front end words there is a bloom and always constant change in
spk_0
struggling to build new tooling that can catch up with the front end complexity.
spk_0
Well the fact that we actually need the visual testing is an indication of how much more
spk_0
challenging front end testing is it's like almost like saying let's test the bits flowing over
spk_0
some network connection and basically try to analyze the system from that if I'm using like
spk_0
a silly analogy. Yeah so you were talking about you mentioned contract testing can you elaborate on that?
spk_0
Oh definitely I think that this is now one of the most interesting fields in testing in overall.
spk_0
I would even say that if in 2025 you're not doing you're coding against API without any
spk_0
explicit contract it shouldn't be an option on the table. First let me explain the risk.
spk_0
So we are all consuming APIs whether front end to backend back end to backend and
spk_0
every system is distributed. Now if as a consumer of an API the payload that I'm sending and
spk_0
receiving for me are just kind of unknown JSON some kind of JSON that I can't control and predict.
spk_0
Then I'm a tourist of receiving a lot of breaking changes so say for example
spk_0
things work but then my APIs that I'm consuming change something my test pass but production will fail.
spk_0
And at this point we are really good at testing our components but that
spk_0
the ocean between components we have much less testing ways and techniques to ensure that two
spk_0
different parties are aligned on the API between the two. And now there is a bloom of new techniques
spk_0
and tooling to cover exactly this. As a API consumer how do I know how do I get a guarantee
spk_0
that the the payloads are changing and I should be prepared for that.
spk_0
So what are the solutions in this space? Yeah so there is a spectrum of tooling and options.
spk_0
The simplest one just in a monorepo just creates zod schemas share them between the backend and
spk_0
the front end and you get a solid protection right I mean whenever the backend changes the schemas
spk_0
your front end test or it can be also your backend test will fail because you have now you can
spk_0
generate typing from this from these schemas you can also do runtime validation and you're protected
spk_0
but it doesn't cover a lot of risk. On the other side of the spectrum the more fancy tooling
spk_0
are mocking servers that will let you know that will provide you with a very sophisticated
spk_0
mechanism to synchronize the two parties but there is the middle I think that most of the solution
spk_0
that are somewhere located in the middle between that are not too complex and not too simple and
spk_0
are not simplistic use the following the API provider generates open API usually it comes for free
spk_0
and it somehow share that open API with the consumer now the consumer all the consumers have
spk_0
tooling to generate both types but also runtime validation out of open API. So now as a front end
spk_0
for example whenever my backend changes the open API if my types I say it's a new field was added
spk_0
my code will generate types types based on this open API and if I'm trying to access some fields it
spk_0
doesn't exist anymore or is the wrong type my I will get compiled time or testing failure right away
spk_0
so it kind of mean backend are always aligned without spinning up a complex and fancy end-to-end
spk_0
environment. Yeah but like you said it almost kind of necessitates in many cases working in a
spk_0
monorepo otherwise first of all I hesitate to call a test I mean it's it's basically one of
spk_0
the benefits of using static typing is that you get certain bugs become impossible because of
spk_0
the type system but I don't really consider it to be tests it's basically that's I might
spk_0
you could call it another step in your build that might fail but I hesitate to call it tests
spk_0
maybe it is I don't know that's an interesting philosophical question. Some of the fancy solution
spk_0
are going beyond static typing so what they do is that if you have an open API it's not all about
spk_0
typing some kind of the assertions there are can be asserted only on runtime say for example you
spk_0
have a reject some of the fields type is a rejects you can't assert this in typing in TypeScript
spk_0
also TypeScript is now adding some kind of reject support so what these tooling are doing during
spk_0
testing whenever you are sending to your mocks on payload they are verifying whether the received
spk_0
testing payload is indeed conforming to the open API definition so in that regard they can
spk_0
make the test fail if what you're using the payload you're using is not what the back and the API
spk_0
defined. Cool for this you need like real testing and maybe it's worth mentioning one work very
spk_0
cool features I guess you both know MSW it's a very popular JavaScript library for mocking
spk_0
intercepting HTTP requests. No sorry I don't. Oh okay so let me introduce it MSW is a library
spk_0
that allows by the way both also Node.js also backend it allows you to kind of define
spk_0
in your in your environment kind of intercept network request and say hey whenever I'm approaching
spk_0
another API say the backend intercept it in browser it using by the way service workers so you
spk_0
will see the request in the network tab and Chrome but instead of going out outside your browser
spk_0
MSW will return the response on behalf of your API so you get a much faster and more predictable
spk_0
working environment. Now MSW added a new feature this year and we're discussing recent changes
spk_0
that instead of you defining what will be the API responses and you might be wrong there right what
spk_0
you think that there might be three fields in this payload the backend in fact in production will
spk_0
return four or two. MSW generates the responses from the backend open API so you provide it hey this
spk_0
is what the API provided us the real word the real schema that was generated and it takes and
spk_0
generates responses based on this so it's kind of you are you and the API that you consume are aligned
spk_0
are always aligned does it make sense yes it does yeah MSW stands for mock service worker in case
spk_0
that wasn't clear but already so before we wrap up is there anything else you wanted to talk about
spk_0
Rokrek that we have not yet covered in this world. Let me see I've prepared a table of content here
spk_0
on the side we can discuss a little bit more about testing with AI also we have we have covered this
spk_0
nothing insignificant as it we we have touched the module points in advancement in testing.
spk_0
All right so if before we I do have a final question though in this context so when you usually are
spk_0
when you're brought into a new project and and you want to before making any changes I assume you
spk_0
want to make sure that you have good test coverage because who wants to start making changes in
spk_0
a project before we have good testing coverage what's the main thing that you're looking for
spk_0
you know to say this project has good test coverage versus another project that you might say
spk_0
this project does not have good test coverage. Yeah that's that's the million dollar question I'm
spk_0
even struggling to enter it it's so it's so deep but I think that I'm using to I'm using two
spk_0
erasics for this the first one is unfortunately not it's not comparing it's not measured if people
spk_0
are moving if people are really are having confidence to deploy fast deploying deploying fast and
spk_0
just feeling confident to do it you see it I mean you you feel it they trust their testing that's
spk_0
I think the ultimate eventually the ultimate test on top of this I'm so it's literally just to just
spk_0
pause it for a second it's literally that subjective feeling that you test you can ask people
spk_0
working on the project how well do you trust your tests if you if you make change and it makes
spk_0
its way all the way to to to to being deployed to production how secure do you feel about it working
spk_0
properly and not breaking any of the things or introducing any form of aggression it's it's
spk_0
essentially that that level of subjective feeling in the team yeah that's the bottom line I'm also
spk_0
using test coverage I'm using branch coverage yeah I know that it's not a perfect metric but it's
spk_0
a very simple metric and for the majority of team they're not like they're not foolings
spk_0
if the if if the system has a very high branch coverage it means that the testing are indeed
spk_0
testing most of that 90 coverage not all of them but a high percentage of this at least it tells
spk_0
me that most of the system areas are are trying to be tested on top of this I usually add some kind
spk_0
of mutation testing so mutation testing means that it's a framework that plant bugs in your code
spk_0
like it can plant a bug in each and every line and then it runs your test and in truth that they
spk_0
fail and if yeah now because I mean it's it's it's an it's amazingly efficient technique to realize
spk_0
testing efficiency but on the other end it's really slow and cumbersome so what I usually do I run
spk_0
it like I run it occasionally just to get a sense of what is the ratio between the code coverage
spk_0
and the mutations so in other words if I see that I have code coverage of 90 percent in a file
spk_0
say a file has 90 percent coverage and the mutation score is very low it tells me that this
spk_0
is a nonsense coverage you know that has a just visit some of your code of it but it's not a
spk_0
certain one anything on the other hand if I see correlation between coverage and mutations in
spk_0
a file I know that this coverage is in authentic it's like trustworthy so this is the I'm
spk_0
peer excited of measuring down the test cool okay thank you for that elucidate elucidation
spk_0
is that how you put it elucidation that's the word I'll Google that word right after
spk_0
now elucidate to explain something clearly is basically what it means so that is the word of the day
spk_0
elucidate all righty so without move on to picks picks are that part of the show where we get to
spk_0
talk about anything we want to within reason of course board games books food movies tech if you
spk_0
want to I'll go first and my pick I do have a pick before I get to the high point of the dad jokes
spk_0
of the week and it has to do with dad jokes so at the end of our podcast episode that Dan and I
spk_0
recorded with Ryan Carniano and Tanner Linsley I found out something that I wasn't aware that now
spk_0
of Crosby who's a founder of a G Grit had died and apparently it's some type of helicopter accident
spk_0
well it's to be fair it's not it's not new it's right I know it's the first I'd found about it you
spk_0
know so yeah that just tells you how out there in the news I am every day um and that brings back
spk_0
memories for me so four years ago we did a job-ish-grip-jabber episode with Nile myself and A.G. and
spk_0
I was just a day or nil and Chuck and the reason that I was episode 504 came out in October of 2021
spk_0
and this was one of my favorite episodes as I mentioned before because when I got to the end
spk_0
Nile just jumped out of the end and threw out a whole string of geek dad jokes um that had to do with
spk_0
going to a no-sequel bar and there's no tables so then he went to a sequel bar that had
spk_0
tables but he couldn't join anybody and it was awesome I was given him a standing ovation
spk_0
so that's my pick of the uh for the week is uh you can find it on top in devs uh episode 504 and
spk_0
if you want to look at the transcript or listen to it it's uh it's one of my favorites so with that
spk_0
we'll get to the dad jokes of the week um so a question how come nobody laughed at the joke about the
spk_0
faulty de-teen it was poor execution that's good all right my wife told me the other day
spk_0
I'm really getting sick of you overusing contractions and I said it's what it's
spk_0
and then uh someone threw a can of soda at me today or I'd say pop on the west coast but I'm all right
spk_0
it was a soft drink and those are the dad jokes of the week and those were definitely dad jokes
spk_0
yes they were I'm embarrassed to admit that I understood two of the three I'm I promised to uh
spk_0
to rewatch that section and uh catch up okay yeah I can explain yeah sort of kills a joke
spk_0
when you have to explain it so I'll explain it later that joke is uh
spk_0
Dan what do you got for us okay I've got a couple of things so first of all I
spk_0
posted it on x but I also want to shout it out a big thanks to uh Theo also known as Theo 3GG
spk_0
who puts out a lot of content online uh I'm working on a project that involves uh
spk_0
stripe integration for one of our services and I've not worked with uh stripe uh significantly
spk_0
I think ever so I was kind of learning a lot about the stripe APIs and how it works
spk_0
and they seem great but Theo's video was essentially all the stripe gotchas that he encountered
spk_0
while working on various projects and how to overcome them uh and it was very very helpful it
spk_0
came along it just the right time uh and uh undoubtedly saved me from a lot of uh pain down the line
spk_0
so thank you for that Theo's thank you for sharing this information this is what sharing on the web
spk_0
is all about so that's one thing I wanted to mention another is that we did this kind of uh watch
spk_0
a long party at work it's something that we tried for the first time and got a lot of good responses
spk_0
it's great when you can have somebody in in at work giving a talk on a particular topic but it's
spk_0
not always possible obviously you may not have experts in house on everything so what we actually did
spk_0
is we wanted to learn more about mcp and jack herrington who we've had on the show a couple of times
spk_0
he was also one of the hosts on uh on the react podcast um and he created a series of videos about
spk_0
mcp that are just excellent so we watched several of them in a row they're they're actually fairly
spk_0
short each they're like 10 to 20 minutes each uh and he explains things so well so we basically
spk_0
watch a video and then had a brief discussion about it and then watch the next video and so forth
spk_0
and it was really great and I highly recommend this uh if you if you want to like educate people at
spk_0
work it's a it's a great approach I think and the final thing I want to shout out is that
spk_0
I recently encountered this video this guy does videos about old hardware and he released a video about
spk_0
personal computer from the early 80s really called the ti 994a and the reason that this
spk_0
hit close to my heart is that that was my first personal computer when I was a kid it was a very
spk_0
weird and wacky machine and he literally kind of opened the screws and showed the inside of how
spk_0
that machine worked to explain why it was so weird and wacky and why it worked the way that it did
spk_0
and what was good about it and a lot of things that weren't so good about it and um you know
spk_0
it brought back a lot of memories so I'll share that link to that video let's just say that
spk_0
it was really interesting to program for a system that had all of 4k of RAM you know in these days
spk_0
of gigabytes you know this is all we had uh yeah that's it 4k of RAM and within that 4k of RAM I
spk_0
implemented a donkey con like game that had two screens and I was like 13 at the time so yeah it was
spk_0
which language did you use I'm curious basic because that's what it had
spk_0
uh it it was uh it i joke that it simultaneously got me into programming and kind of also
spk_0
scarred me for life so my first job in tech was doing tech support and we were dealing with
spk_0
windows 3.1 with our software and I was constantly having to help users with memory issues and
spk_0
upper memory and lower memory and shifting things around so that things could run and
spk_0
and then windows 95 came along and a lot of that went by by i have a whole story about that but we
spk_0
don't have the time we could probably do an entire episode on reminiscing uh so those would be my
spk_0
picks for today all right yani or to you yeah I recently found myself uh I'm interested in
spk_0
quality coffee so I really I mostly in the coffee story so I really I always thought that like an
spk_0
expensive coffee is like expensive wine you just pay for you know for that brand or for that weird
spk_0
test but then I when I started reading more and more and see videos about testing I realized that
spk_0
like between the farm and your cap there are like 10 mistakes that says that like like
spk_0
dozens of mistakes some some most of them are intentional intentional to just to just cut
spk_0
coast and make the coffee horrible take for example in many commodity farms when they uh when they
spk_0
pick the the cherry when they when they collect the cherry picks some of them are already
spk_0
just mature and some are unriped where some are red some are green uh so but you know they have
spk_0
to be efficient so they just take the entire tree down you get coffee beans that are all right unriped
spk_0
together some are really sour some are really really sweet and and that's just one thing that the
spk_0
coffee um coffee production is doing in order to bring you coffee with lower cost and more revenues
spk_0
for them and really getting a great cup of coffee demands a lot of a lot of uh decisions and optimization
spk_0
all over the all over that process and it's um yeah it's uh there's no point here it's just a very
spk_0
fascinating uh journey that involves biology nature uh people yeah other than that I
spk_0
I realized that we are becoming more and more dependent on LLMs right I mean we're we're about to
spk_0
use it for 10% of recording or maybe 90% I mean yeah it's objective uh and um I felt like it's
spk_0
too much of a black box for me I can't understand the mechanic then so say for example I'm providing
spk_0
it with 15 instructions it respecting 30 why it fits inside the context window why did you ignore
spk_0
20 of them uh so I bought a book which is called I wanted to understand this um this box from from
spk_0
inside I bought a book that's called building an LLM from scratch it's um just teach you build all
spk_0
the layers of an LLM of course a very simplistic one but you get the idea of all the moving pieces
spk_0
I'm now really eat coding eat uh and I think that I uh after professing for chapter some I can surely
spk_0
recommend it to go back to your first pick I would all I will say is that the term quality coffee
spk_0
is an oxymoron so there is no such thing and I'll leave it at that so all right well thank you
spk_0
for joining us today and talking about testing I appreciate it you coming here uh before we go if
spk_0
people want to get hold of you and give you money and and uh do all that kind of stuff where is the
spk_0
best place to track your genius thanks for mentioning this well my my gita page has a contact there
spk_0
is also goldbergionny.com uh and you said that you work as uh as a consultant so I assume that you
spk_0
give assistance training and help organizations improve their testing and test coverage and
spk_0
quality of tests and whatnot yes workshops um and on and on and all being any any any kind of
spk_0
uh assistant and training related and that's it yoni goldberg goldberg. Goldberg. Yoni. Excuse me
spk_0
already and yoni is y-o-n-i just yes you know or gita or gita.com slash
spk_0
goldberg. Yoni okay already well that is it another thrilling episode is in the book thank you
spk_0
for joining us and we will talk to you next time on job scoop jabber.