Matt: 0:01 What's up, wizards? I am here with Mark Erikson, who is a legendary maintainer of Redux, extraordinary coding brain, my personal friend, and someone who I've known for quite a while in the TypeScript and open source community and someone I admire a lot. I'm really excited to have you on, Mark.

0:23 The thing I really want to talk to you about is you've been working on an open-source code base at your current role. Could you tell me more about what you're up to and give us a potted intro to Mark?

Mark Erikson: 0:35 Sure. A year ago, I joined a startup called Replay. We're building a 1 time-traveling debugger for JavaScript applications. One of the many things I love about working at Replay is that our client-side code base is entirely open-source.

0:56 In fact, it actually started as a fork of the Firefox DevTool, like Firefox browser DevTools folder. We basically turned that into an application. Three [inaudible] of source. It's public. Anyone can browse. Anyone can submit PRs, if you really want to. All the work that I've done on that in the last year is public, which is really cool. You can actually look at all the PRs that I filed.

Matt: 1:22 It's amazing. When you joined, what kind of state was the repository in? What did you come into the job to find?

Mark: 1:33 It was interesting. Like I said, the application started as a copy-paste of the DevTools folder from the Firefox browser as it existed in about 2020. The Firefox DevTools are a React-Redux application.

1:48 Most of that code was written back during 2015, 2016. Very early-stage React and Redux code. The way you write React and Redux has changed drastically in those years. There was class components versus function components, in some cases uses of React.createClass, no TypeScript, anything like that.

2:13 In fact, even the Firefox build system didn't allow use of Babel early on. In some cases, instead of using JSX syntax, there was this ancient like React DOM factories thing where you manually code functions called div and span, etc.

Mark: 2:33 When I joined last year, the code base was about a hundred and forty, hundred and fifty thousand lines. About 80 percent of that was still the old Firefox Dev tools code with a few changes, and about 20 percent of the code base was relatively new with replay-specific features added onto that. Implementing features like the list of recordings library, the authorization code, stuff like that.

3:02 The new code was mostly TypeScript. That mostly followed relatively modern TypeScript usage, React and Redux function components, all that stuff. There was a huge split in how the code base had been written.

3:20 For that matter, the Redux code was very old-style. It was still a lot of handwritten action-type constants and object spreads and bad patterns like putting maps and sets in the Redux store and all that stuff. I knew that going in because the code base was open and I'd had a chance to look at it. I knew I was going to end up spending a lot of time helping to modernize this code base.

3:47 Over the course of the last year, those of us on the front-end team, primarily myself and Brian Vaughn, have done a ton of work to modernize, rewrite, refactor, revamp, whatever terms you want to use. The code base is now down to about 90,000 lines. It's like 95 percent TypeScript, and we've drastically improved every aspect of it so that we have a good foundation going forward.

Matt: 4:16 A lot of your work then has been refactoring, I guess, doing some new features, but mostly focusing on just [inaudible] .

4:21 [crosstalk]

Mark: 4:22 Yeah. In fact, we really finally finished up that refactoring and rewriting process getting into December. We're recording this in mid-January. It's really just within the last few weeks that we've really been able to start turning our mindset and our time towards working on actual new features.

Matt: 4:40 Wow. Basically, the code base you opted into or the people there before you opted into a legacy code base, essentially, to pull in all of these features that they had, built stuff on top of it, and you've come in to corral it and turn it into TypeScript as well. On your side then too, because you've been doing that, but also in your spare time, you work on Redux, right?

Mark: 5:04 Yep.

Matt: 5:04 I guess Redux has been through a similar transformation in terms of TypeScript coming into the project. Is that right?

Mark: 5:12 Yeah. I took over as Redux maintainer in summer of 2016, about a year after it came out. I actually wrote a whole long blog post about how I ended up learning TypeScript. I wrote it at the tail end of 2019...

5:31 If you look at it, I expressed frustration because, throughout 2017 and 2018, I was getting a lot of issues being filed on the various Redux repositories complaining about our TypeScript types in various situations. I couldn't do anything about those myself because I didn't know TypeScript.

5:53 It was frustrating to see people raise complaints about these things and not be able to do anything about them because I literally didn't understand enough to make any relevant changes or offer a response. This actually got really weird because I was working on this new, at the time, Redux starter kit package, which we later renamed to Redux Toolkit.

6:19 The original alpha versions were all plain JavaScript. Later on, a contributor converted the code base to TypeScript. There were a few months where I really couldn't even make changes on this library that I was trying to build because I didn't have enough TypeScript expertise.

6:38 Fortunately, Lenz Weber, who is now one of the Redux Toolkit co-maintainers, got involved, started filing some PRs to improve the Types, and basically ended up helping teach me TypeScript through the PRs that he was filing, which was a really unusual position for both of us.

Matt: 7:00 What a guy to learn it from as well.

Mark: 7:02 Yeah, Lenz is an absolute wizard at TypeScript. Occasionally too much so for our own good, but he's done some amazing things with our Types. I'm now at a point where I can do some reasonably advanced library-level TypeScript tricks myself, although I'm still flailing my hands trying to make it work and getting lost in the process.

Matt: 7:27 That's really interesting. At the same time as you were doing or I guess, Redux has been going a long time for you, you upskilled in TypeScript there by transforming Redux Toolkit and Redux over into TypeScript. Then how did that affect your experience of transforming Replay then? Did you go on to feel more confident?

Mark: 7:47 Yeah. In fact, Replay is actually the second or third application codebase that I'd migrated to TypeScript. At my previous job, I'd worked on a Backbone app that we later were beginning to migrate to React. We began throwing TypeScript. That was the one where I started learning how to do TypeScript myself in 2019.

8:12 I'd set up a couple of brand new great React app projects with TypeScript. Then later on, 2020, 2021, I ended up taking over a classic MEAN Angular 1 codebase, and then we migrated that whole thing to React Next in TypeScript. I've had a lot of experience doing that migration at this point.

Matt: 8:36 Got you. Let's start there then. What is your mental model when you start a migration? When you're thinking, OK, we have this JavaScript codebase here. Let's say that, because it sounds like with each of your migrations apart from the Replay 1, you've been transforming framework as well as language, essentially.

8:54 You've been switching from Angular 1 to whatever, like Backbone to React. From a language point of view, from TypeScript point of view, what's your mental model? How do you start that migration?

Mark: 9:08 Step one, got to make sure the build tooling is in place. The very first thing I did on the Angular app was upgrade all the build tooling, getting Yarn in place, dropping Bower, actually brought in Create React app and horribly mangled the webpack config just because I knew it was a thing that would give me good output.

9:25 Step one is set yourself up for the ability to use TypeScript in the codebase at all. Pick some tiny utils file that doesn't have any other dependencies and make that the first thing you convert just to prove that you can compile it.

9:48 If you've got bad types in there, it'll error. You can convert at least one file and build the project and detect errors, if there are any. Get the build tooling in place, to begin with.

Matt: 10:03 Can I jump in on that one, which is that's really interesting that you say that particular type of file. Because if you think about your application, it's basically a graph.

10:13 You have leaf nodes, which is your components, which tend to depend on a lot of things. You have things that are further towards the top of the tree. You're thinking that you should start further towards the top in when you migrate or...?

Mark: 10:27 It's different things. Literally the first step is make sure that we can actually use TypeScript, whatsoever.

10:37 My thought is by picking some little utils file, because that file isn't going to depend on anything else, because it is one of those leaf nodes, it's like the tiniest thing you can convert without getting into a mess of having to modify a bunch of other files in the process. Step one is just, does TypeScript work in this project?

11:00 Then, once you've proven that, the next step would be actually start to look at your core application logic in some way. What are our data models? What are API calls? What are some of the key data types that are flowing through the system?

11:22 Then start to get those typed out, even if the code that's using that isn't typed yet, because once you start converting your components or other kinds of logic, you're going to be needing those types everywhere.

11:39 The earlier you can start to define the correct TypeScript types for your data, that's going to have a ripple effect on the rest of the system.

Matt: 11:48 Interesting. That's hard stuff as well, like figuring out those decisions. You're saying tackle that early, basically, front load that work. Was there anything in Replay that was difficult to convert in that way? Were you involved in that process in Replay or that started before you got there?

Mark: 12:09 Like I said, there was TypeScript in the code base, but it was primarily all the brand-new code. In terms of migrating the existing code to TypeScript, I did about 95 percent of that work.

12:21 Some of it was very, very intentional. I'm just going to dedicate a couple more weeks, more chunk of time to doing that. Some of it was like, "OK, I finished a task, and I've got a couple hours left in the day. I don't feel like starting anything else. Let me just pick a couple files and migrate them."

12:41 My brain isn't fully working today. I don't feel like I can actually focus on a thing, but I can migrate to TypeScript basically in my sleep at this point. I'm just going to mindlessly convert another three or four files.

12:55 My CEO even actually made fun of me occasionally about this, but it paid off. It's paid off hugely to where...We actually had tons of dead code in the code base.

13:10 There were features that the actual Firefox DevTools had that we literally were not using or chunks of code that were technically still getting loaded but didn't make sense for Replay, the application.

13:24 Trying to track down the dead code was much easier when we could confirm that this file isn't actually being used anywhere through TypeScript or trying to figure out where a piece of code is being used, so you can refactor it and being able to right-click, find references, and actually have confidence that you've found all the usages, which is the kind of thing that TypeScript can give you.

Matt: 13:54 Interesting. The first phase is make sure TypeScript works. Then, convert over your core business logic. Then, do you end up in the sleepwalking phase?

Mark: 14:07 Pretty much, yeah. Converting a React component to TypeScript, converting more utils or whatever, that's almost always really straightforward. Yep.

Matt: 14:18 I want to ask you then a question about...When you express types, when you're thinking about the core business logic in your application, often what I find is you end up making a lot of decisions about what domains exist in your application.

14:36 Let's say that you're generating types with Prisma, let's say, and you have a user type that comes from your database. If you have an app header type, let's say, that you're using a nav bar, and you've got a little avatar there, should that avatar component be using the type that's generated from Prisma?

14:57 You have this dependency going through your application then between your types. I wonder if you have any kind of thoughts about that and whether you were thinking about that in terms of Replay.

Mark: 15:07 Kind of. In terms of Replay, for the most part, I was following the existing structure and adding types to things that were in place. In fact, that Mozilla Firefox source code, once upon a time, was actually typed in flow.

15:37 The Firefox Devs had actually stripped out the flow types at some point prior to us copy-pasting the code. There were times I was looking at files and I'm like, "I don't know what all the proper fields are in this object that's being passed around.

"15:53 I'm going to go back in the source code history for Firefox, find a version of the file before they stripped out the flow types and either manually rewrite some of those types or actually copy-paste the flow types into a converter to convert them to TypeScript." In that sense, a lot of it was just going along with the existing structure.

Matt: 16:13 Yeah, I see. So not having to think too hard about the structure, because it's a big legacy app. Let's the structure's all in place. Yeah, that makes a lot of sense. Let me think actually. We could start jumping into code now if you fancy.

Mark: 16:26 Sure.

Matt: 16:27 Obviously, we can look at a whole load of different things. I was jumping around the Replay DevTools earlier, and there's a whole load of interesting stuff happening in there. I'm sure some of which you've contributed to and some of which you haven't.

16:43 Because I'm really interested in that idea of the core types, are there any particular files inside Replay that have a section of those types, let's say?

Mark: 16:54 Let's see. The code base has also changed a lot over the last few months, largely because Brian [indecipherable] has been rereading a lot of our features. In fact, you'll even notice that a lot of the functionality now lives under the packages /replay/next/src.

17:15 Brian has been making a very intentional effort to separate out a lot of the new features for various reasons, both conceptually, architecturally, trying to keep the old code from leaking into the new code. Yeah.

Matt: 17:34 This is actually really interesting, this stuff here. This is in Replay next/src/utils folder. There's a bunch of stuff I can see here already, like array.ts, a bunch of interesting types here. Let's stick with this sort of thing.

17:53 For instance, we've got this findIndex function, which we can see what it's up to. What I'm interested in here is I've chatted to Orta Therox for one of these. He made a distinction between library-level code, in terms of TypeScript, application-level code, and then utils folder code.

Mark: 18:15 I haven't heard that third distinction before.

Matt: 18:19 I'm really interested in your thoughts around that, library-level code being the most complex, utils folder code being relatively complex sometimes, like these findIndex functions, and then application code being the consumers of TypeScript.

Mark: 18:36 To a certain extent, we could think of utils style code as being like library style code. It's just internal and not actually published. The way I think about it is that for the most part, application-level code tends to be...It's relatively simple. You're declaring basic object structures.

19:05 What are the types of my API responses, data coming back from the server, my React component props. You probably have general function arguments, whether it's a callback function that's being passed around or something like that, but for the most part, those are relatively simple TypeScript types.

19:29 You might have a few generics or something in there, but for the most part, you don't need too much more than basic interfaces, basic types, maybe some unions of possible arguments for a React prop or something like that.

19:48 What I think of as library-style code, you're having to imagine ahead of time, how is someone else going to use this code? What kinds of problems are they going to need to solve? What kind of data are they going to be passing in to these methods that I'm writing?

20:06 That's where a lot more generics come into play, because maybe you do have a find index function that can be passed an array of any kind of thing. It doesn't actually matter what the thing is exactly. It just matters that we are passing an array of this item and maybe returning the one item or its index or whatever. Or, in the case of Redux Toolkit, what is the initial state of my slice? What does the reducer function return?

20:34 That's where you have to start thinking ahead of time, "How is someone going to use this?" You're thinking both in terms of the API design and what kind of data has to flow through this system. Your mindset to writing that code is very different. The TypeScript approach for writing that code tends to be very different.

Matt: 20:57 Then the utils folder stuff, it feels to me like a crossover area. I'm really interested in how you felt your knowledge of the more advanced stuff has influenced your design and your ability to contribute to that kind of code, the higher-level, application-level code.

Mark: 21:16 Actually, I can think of a specific example from the Angular app that I worked on a couple years ago. Very, very short background. It was an internal project metrics and statistics database. Projects across the company had a mandate that they had to upload stats about their project every month.

21:35 Our back end would take these uploaded Excel files, extract a bunch of info, and do some post processing to spit out red, yellow, green statuses in various categories. The app had about 30 different metrics that it knew how to extract and process, each of which required storing several different pieces of data.

21:58 It had to have logic. It had to have similar logic to do post processing, but unique to each of those metrics. The previous lead developer that I took over from had defined a fairly consistent pattern for doing this work, but, again, it was all plain JavaScript.

22:18 When I took over, it took me months to try to understand what all that code was doing, both conceptually and the data types. There was one particular chunk of that that I really wanted to be able to define, again, a couple of those key data types that were going to flow through the system.

22:41 It was almost like an OOP-style override thing, except with plain data, where there's going to be a report type, which always is going to have a certain...It's got the date and the name and then a contents field that is going to be unique to that report type for that metric.

23:03 I wanted to be able to make that content field generic, and then declare the processing logic and just say, "Here's what the contents field type is going to be," and have that ripple through all the processing functions so I didn't have to keep redeclaring it and just have the rest of that inferred.

Matt: 23:21 Perfect.

Mark: 23:22 Writing that processing logic and its types probably took me a week or two to try to get all the inference set up correctly. Once I did, it was beautiful. You just go to the file, you declare the interface or the fields for the content type, and say, report, angle brackets, my content type, apply that to the processing structure definition.

23:48 Later on, you define the handler callback and it knows all the types. It just flowed all the way right through. That was basically internal library-style code in the application codebase. The mindset for writing that code was very different than going off and writing yet another component.

Matt: 24:09 That's fascinating because that reduced a lot of duplication, I can imagine.

Mark: 24:15 Oh, yeah.

Matt: 24:18 Did that feel easier to maintain going forward as well?

Mark: 24:21 Absolutely. It gave us a lot of confidence knowing that as we did have to start adding a new metric definition, we had a very consistent boilerplate-y but intentionally so pattern. Of course, we always usually started it by copy-pasting an existing file.

24:40 You knew you just define the content field, declared a report type, pass in the generic, declare the processing structure, pass in that generic, and all the rest of it's going to be typed correctly.

Matt: 24:52 Fascinating. It's I guess when you find the right abstraction, when you find the right level that you want to operate at, you find the generic that matches that, you understand the flow that goes through that, then you get the difference, right?

Mark: 25:07 Mm-hmm.

Matt: 25:08 Fantastic. Then let's think about Redux Toolkit because there's a lot in Redux Toolkit that has a lot of these same demands, right?

Mark: 25:19 Yep.

Matt: 25:19 If we open up some of the code and you wanted to look at reselect, I'm actually unprepared for this, so you're going to have to guide me a little bit, what am I walking into here?

Mark: 25:32 Reselect is a separate library, but it's always been considered part of the Redux family and is normally used with Redux. It's a library for defining memoized selector functions. The idea is you write a couple of functions that extract data from something like the Redux state, and we only want to recalculate the output when at least one of the inputs has changed.

25:57 This is useful for improving performance because you can skip doing calculations unless they're actually needed. It also is useful with React Redux because it needs to check have the references from these functions changed in order to determine if your React components actually need to rerender or not.

26:22 Reselect was always maintained by someone else, but it was in the Redux GitHub organization. The previous maintainer had drifted away. In early 2021, I had noted that there would have been a bunch of complaints that had piled up over time about things that weren't really feasible with the API.

26:45 I filed this giant discussion issue and said, "OK, here's the list of all the complaints. Here's a bunch of other libraries in the ecosystem that do similar things. How can we make this better?" I basically took over as the maintainer, and in late 2021, I began working on trying to improve reselect.

27:04 One of the first things I did was I took someone else's PR that they'd filed to improve the TypeScript types and began trying to apply that. For a point of comparison, scroll up under src. If we expand the legacyTypes folder, there is a ts4.1 folder with a index.d.ts file in there. If you jump down to the bottom of this file, the very last line, you'll notice that it's actually 3,000 lines long.

Matt: 27:38 Oh my God. I don't have line numbers on here. Oh, hello. [laughs]

Mark: 27:44 The reason for this is that these TypeScript types were handwritten separately. The original code was plain JavaScript and the types file was in the repo, but handwritten. Because you can pass in an arbitrary number of input functions, it had to have the types to extract the results of each of those and apply those to the final generated selector function.

28:12 Unfortunately, with a lot of languages, the only way to do that -- I've seen examples of this in C# -- is to define N overloads of the function with one to N different overloads for the numbers of possible inputs.

Matt: 28:32 Let me just get my bearings here. We have a massive function overload here. I'm counting 108 counts here.

Mark: 28:43 To make it worse, you could pass in, say, five input functions as either separate arguments or an array, which means we had to have double the overloads to handle the arguments versus the arrays.

Matt: 29:03 Now we're at the array arguments. These are expressed with tuples instead. Good God.

Mark: 29:10 This file is like 3,000 lines and utterly, completely unmaintainable. Fortunately, when I took over as maintainer, someone had filed a PR that had been sitting around for months to try to change these types and use some of TypeScript's improved inference features instead.

29:29 I spent much of November, December 2021 merging that PR. Then it broke things because I put that out as a 4.1 release. I was trying to avoid breaking changes. I spent weeks iterating on those types. I was often over my head. I know what I want to do, but it's too complicated for me to get done myself. Let me beg other people for help and somehow coerce this to work. Where we ended up...

Matt: 30:00 Can we take one step up? I just want to demonstrate what this inference looks like, if that's OK.

Mark: 30:06 Sure.

Matt: 30:06 Just so we get a sense of what you were trying to work with and how flexible this was. If we have a const object, let's say it's like a, 1, b, 2, for instance. We want to say const selector = createSelector, because I'm very familiar with this because I use this back in a day. Let's see.

Mark: 30:29 The first thing you probably want to do, especially with our current types, is define yourself what we would usually refer to as a RootState type. If you're using Redux, this is the type of the Redux state. Make that to say a, number, b, number, whatever fields you want to put in here.

30:56 The way reselect works is we can pass in one or more what I usually refer to as input selector functions. These are later going to get called with whatever values you pass in. Their job is to extract something from those arguments.

31:16 Later, when you call the function and it looks at the extracted values, if any of those extracted values are a new reference, then it will run the final function, which gets those as arguments, does a transformation, and returns the final result, but only if the inputs changed.

31:38 In this case, let's do a really stupid example that's just going to extract these two numbers and adds them. A real example would likely create some new references, but this will show the syntax.

31:49 Inside of createSelector, let's pass in a function that's going to take a value of type RootState as its argument. Let's just return state.a. Let's copy-paste that and do the exact same thing, except that we return state.b.

Matt: 32:20 We got our aSelector and let's say we got our bSelector, and this will return b.

Mark: 32:27 A lot of times, these get defined in line. It doesn't matter as long as the...Hang on. If we're going to write these separately, let's remove the createSelector bits from them. These are just standalone functions. Now let's write another function called selectTotal, or something like that, = createSelector. We'll pass in aSelector and bSelector as arguments.

32:57 Now we'll write a third function and we'll just declare it in line. It's going to take two arguments. Let's just call them a and b. You'll notice if you hover over a, it is correctly typed as a number. We didn't declare that. How did we get it?

33:21 That's because the types for createSelector have looked at each of the input functions and it said what is the input, what are the parameters for the function, what is the return type of the function.

33:37 The return type of aSelector gets applied as the type of the first argument to this output function we just defined. The type of the return type of the second function gets applied as the type of the second argument, repeat, repeat, repeat, ad infinitum.

Matt: 34:01 This ends up being a function, which takes...We've got a fair bit of stuff going on here, but it looks like a function that takes a RootState and returns a number.

Mark: 34:14 Exactly. We've collectively looked at what are all the input function arguments. We have to do a lot of work to make sure that they match up OK, because what if bSelector took a completely different object type as its argument? Would that be legal or not? Maybe those clash.

Matt: 34:39 Interesting.

Mark: 34:42 Then we look at what is the final result of your output function, and we mash those two together to figure out here's what's legal to pass in, here's what comes out the other end.

Matt: 34:54 That's fascinating. I want to ask you a couple more detailed questions here. When I hover over this createSelector, I can see that what's being inferred in the type arguments is really detailed. We, first of all, have -- it looks like a tuple -- a tuple with both of the...

Mark: 35:14 The input functions.

Matt: 35:16 Yeah, both of the input functions are in there. Then you have a number at the end. What's going on there? How are you inferring those in a tuple?

Mark: 35:25 This is where the magic comes into play. On the left side, expand the versionedTypes folder and click on ts46-mergeParameters.ts. You are not expected to understand this.

Matt: 35:46 [laughs]

Mark: 35:46 This is my pride and joy. This is the piece of magic that it took me weeks to cobble together where I didn't know what I was doing half of the time.

Matt: 35:58 This single type?

Mark: 36:00 Yes. At a high level, what this is doing is a types level array.map transformation. We're taking in an array of things, and for each item in that array, we are transforming it somehow, and we're spitting out a new array at the end, except that we're doing it at the types level.

36:33 The inputs are an array or a tuple of all the input functions that we're passing to create selector. For each of those input functions, we need to extract the type of the parameters, and there could be many parameters to each of those input functions.

36:54 Then, we need to shuffle it so that we get an array of all the arguments at index, all the arguments at index 1, all the arguments at index 2. Then, we need to figure out, are these even compatible with each other?

37:17 If one function says it needs a string at index and another function says it needs a number at index, you can't pass both of those simultaneously, because later on, when you call the function that you've generated, the first thing you pass can't be a number and a string at the same time. That should be an error.

37:40 If something says, like, "I need an object with an A field at index," and another function says, "I need an object with a B field at index," you can pass an object that has A and B.

Matt: 37:54 Got you. You're intersecting those.

Mark: 37:56 Exactly. We transpose these to get all the parameters at the same index, and then we have to intersect them together to figure out what is the tightest legal type that can be passed at that index.Then, we have to untranspose it and get it back to an array of , 1, 2, 3, so that we know what the final arguments are for the generated function.

Matt: 38:28 Oh, my gosh.

Mark: 38:28 This took me weeks to put together, and I was repeatedly begging for help with it.

Matt: 38:36 Just so we understand where this is being used, then...Sorry to interrupt you. MergeParameters...

Mark: 38:42 [inaudible] be used in createSelector, and we're looking at all the input functions in doing this transformation.

Matt: 38:49 Got you. Inside here, we have basically an interface which represents the resultSelector. Is it createStructuredSelector or createSelector?

Mark: 39:04 It gets used in createStructuredSelector, but we actually want the other one in src/types.

Matt: 39:09 Types inside here, createSelector. Whoops.

Mark: 39:15 Let's see.

Matt: 39:16 Where are we at here? Let me find references.

Mark: 39:26 There we go. GetParamsFromSelector uses it.

Matt: 39:34 Got you. GetParamsFromSelectors. I see. That's the thing that's being used here. I see. OK. OK. OK.

Mark: 39:43 We pass in an array of selectors. We're going to then run this type that looks at them and extracts the parameters.

Matt: 39:52 Wow. I've got to say, Mark, I didn't realize you were this level of wizard. This is one of these defining problems that can basically make or break a library, right?

Mark: 40:04 Oh, yeah. Yeah.

Matt: 40:07 What did you notice after getting this right, after solving this problem? What changed for the library? What changed for the library? What changed for how people were using it?

Mark: 40:16 That's the great thing. In a lot of ways, nothing changed for the users, because I intentionally tried to do this in a non-breaking changes way. In fact, the biggest thing that people had to change with their existing code was there were times that they were calling createSelector and passing in a generic.

40:34 That actually was now the wrong way to use it. It's like, "Don't pass in a generic. We will just automagically infer everything from this." Just make sure that your input selectors have the right types.

Matt: 40:48 You weren't accepting type arguments or anything to...

Mark: 40:53 That's a big lesson that I learned from the [indecipherable] and Redux toolkit is that library code works best when the user really doesn't have to supply anything other than a couple basic function types, and it can infer the rest of it.

Matt: 41:07 Yeah. Which is different from how you solve the problem in the Angular application, right? Because you're actually accepting a type argument, and that was part of the things that you're accepting, essentially.

Mark: 41:18 I actually want to get back to the carrier, because there's one other fun problem in here.

Matt: 41:22 Oh, you're scaring me.

Mark: 41:24 Oh, yeah. As a library maintainer, we try to support a reasonably wide range of TypeScript type of versions, because not everybody is updating TypeScript all the time.

41:37 We've set TypeScript 4.1 as a minimum version that we say we work with, for example, because that's when TypeScript added the really nifty string manipulations and being able to say like, ""If a field name A goes in, a field name capital A goes out," and that will be typed correctly.

41:59 We try to write our types in such a way that they work with, say, TypeScript 4.2 through TypeScript. 4.8 or something. We actually try to compile our type tests and run them in CI against a range of TypeScript versions to catch any possible errors.

42:20 It turns out the TypeScript team does that, too, and they check against a fairly large list of libraries in the ecosystem. A few months ago, while they were working on TypeScript 4.9 alpha, the TypeScript team made a tiny little tweak, and it broke one library reselect. It broke one type, this giant mergeParameter [inaudible] so much timeline.

42:45 [laughter]

Mark: 42:48 They tagged me on that PR to ask, "What does this type actually do?" So I explained it. This is where it gets amazing. Anders Hejlsberg, himself, the creator of TypeScript, looked at my description, and he wrote in a comment, a new type that did the exact same thing, and it was simpler.

43:16 If you look at the other file, ts47-mergeParameters, this type does the exact same thing, and it's way shorter, but there's a catch. It only works with TypeScript 4.7 and greater.

Matt: 43:31 Wow.

Mark: 43:31 I still want it to support TypeScript all the way back to 4.2. Now, if you look at this package, we were already shipping two entirely different sets of types, marked with the types version state.

43:50 If the root package.json says, "If you're using TS 4.1 In earlier," we're actually still going to use that really old giant 3000-line type definition.

44:01 Everybody else gets the current one. Now I had a problem because I needed TypeScript 4.23 or 4.6 to get the really complicated mergeParameters type, and I wanted TypeScript 4.7 and later to get the newer mergeParameters type.

44:17 I didn't want to have to ship a free entire copies of our types. That would have just been ridiculous. Lens had come up with a trick that we have used on Redux toolkit for a few versions. That is doing some development versus build time shenanigans to make this work.

44:39 If we look at this versionTypes four, and you will get index.ts, you'll notice that this is importing the insert the 4.7 one as a default, so we're using that newer version in development, apparently. You'll notice that there's what appears to be a package file in here, but it's not actually named as package.json.

45:04 It has a types version flag in here, and it's pointing to the two different files for 4.7 in later and 4.6 in earlier. It's not being used in local development.

45:17 If you look at the build steps for this, we call TypeScript. We compile we like output to a dist folder or whatever. We're going to have these two TS files converted to .d.ts. Part of the build step is we're going to copy that package file over and rename it.

45:42 In the published package, TypeScript is going to recurse down into this folder and intelligently pick the right type based on what version of ts it is. It's as a build-time, publish-only hack.

Matt: 45:56 That's incredible. I've never seen this before. I did not know this is possible.

Mark: 46:03 [inaudible] . I just copy-pasted it.

Matt: 46:04 What a story, though. You had this incredible type that you fought and bled for. It breaks TypeScript itself. Anders Hejlsberg himself comes up and makes this beautifully elegant one.

46:26 I would love to dive into this with you. I think, though, I will struggle to catch up. I'm going to do some research on this in my own time and make a long-form video where I dive into this myself, because it's absolutely fascinating. I wanted to ask you one more thing.

Mark: 46:45 Sure.

Matt: 46:46 Now we're still in the code. You mentioned something about testing these types. That's a question I get a lot, is how do you write tests for your types in library code? How did you test this?

Mark: 47:01 Just like with actual unit tests, where you run the code with a reducer function, you pass in state and action and you verify that the output value looks like what you want. You can test types the same way and it's "Does this compile as expected? Are there no errors?" You can do that especially by asserting that a certain type is what you expect.

47:27 If you look at the folder on the left, in this repo, it's a folder called TypeScript test, and there's a test.ts file. This was especially a really big part of doing that migration from the Reselect 4. and that 3,000-line types file to Reselect 4.1 with the modernized type definition, and then later on the merge parameters thing.

47:55 If you look through here, there's just several thousand lines of code that set up dozens of possible scenarios. We try to check, "OK, if I pass in these things, does the generated function have the correct argument types? Have the correct return type? If I pass in something wrong, does this error?"

48:17 If you scroll down towards the bottom, there's even some sections that refer to specific issues that users have filed and trying to say, "OK, here's a case that a user ran into. Here's the example code they gave. Does it handle their case correctly?" This file does not run. It gets compiled.

48:41 The type test is just, "Hey, TypeScript compile this file. Does it compile without errors?" In the exact same way that a unit test verifies the expected behavior and documents it, this type test file verifies the expected behavior and documents it.

Matt: 49:01 Fascinating. You have a custom tsconfig in here that moves the path to reselect to just the source index, essentially. It's so simple. Then each one is just inside a function. You're not using any...

Mark: 49:17 Function or curly braces. It's been written in different ways over time. I find this slap it inside the function or something just to separate it.

Matt: 49:26 It's not using any test framework. It's literally just a TypeScript file with, looks like, no exports or very few exports that just...

Mark: 49:34 There are some utils in here. Again, Lenz wrote a lot of these, like expectType. You say, as a generic, "Here's the type that I think this should be." Then you pass in in the code a value.

49:51 Again, this is not running, but you pass in a value. TypeScript will look at it and say, "Well, here's the type of that value." Then this expectType thing is saying, "OK, here's what we said it should be. Here's the inferred type. Are these the same thing?"

Matt: 50:09 So simple when you think about it. You just have a couple of little helpers here. It looks like you're using ts-expect-error quite a lot as well. That's your core stuff handled. Something I always wish TypeScript could do is you could expect a specific error...

Mark: 50:28 I would love to have that because there's been a bunch of times where I had an expect-error on something, and later on, there was a real problem there that was a different error than the one I thought it would be and it was getting covered up.

50:44 In the same way that you can have eslint-disable-next-line name of rule, I would love it if that expect-error could be expanded to expect a specific error code or name or something.

Matt: 50:57 Totally. This is the setup. It looks like you just...Do you have a CI setup on here? Github, workflows, build-and-test-types.

Mark: 51:07 Yep.

Matt: 51:06 You just run it on different TypeScript versions based on the matrix.

Mark: 51:13 Matrix wrap.

Matt: 51:14 Beautiful. Somewhere down here, yarn add dev, and then test:typescript. Beautiful. Simple.

Mark: 51:21 All the Redux repositories have some variation on this approach. The file names and paths tend to differ a little bit just because it's been different repositories over time, but same basic approach in every repository.

Matt: 51:36 Amazing.

Mark: 51:37 In fact, like as an example, with React Redux versions 7 and earlier, the library was written in plain JavaScript and type/react-redux package was maintained by the community.

51:51 For React Redux version 8, we migrated it to TypeScript. We did that by copy-pasting the types from the types package. Then I copy-pasted some of the Definitely Typed type test files and contents over into our repo as a starting point and then added more examples and type tests from there.

Matt: 52:15 So nice.

Mark: 52:17 We were able to ship that with very few breaking types changes other than intentionally changing various options or whatever.

Matt: 52:26 Amazing. Mark, this has been absolutely out of this world in terms of you've gone everywhere that I could possibly want you to go. We went deep into migrating Typescript, deep into the deepest possible disgusting library types you could possibly imagine. Thank you so much for all of this knowledge you've shared with us.

52:50 Is there anything you want to plug or promote? Where can people find you on the Internet?

Mark: 52:54 Two relevant blog posts. I put up an article on my blog in 2019 called "Learning and using TypeScript as an App Dev and a Library Maintainer." This was at the end of my first year really doing TypeScript stuff. It's both a personal journey describing some of the pain points I ran into along the way and then a set of conclusions at the end from both mindsets.

53:20 All the conclusions that I drew then are still entirely valid in terms of the separation of app types versus library types, how type forces you to think, how it forces you to design library APIs. I think all that is still entirely relevant.

53:36 Then I did a pre-recorded conference talk last year for TypeScript Congress, called "Lessons Learned Maintaining TypeScript Libraries," where I talked about some of these tips and tricks.

53:48 Nobody teaches this stuff. This was all very hard-earned, practical. I have no idea what I'm doing. I'm going to Google and read other libraries and ask people smarter than me and beg for help repeatedly.

54:06 I think I can fairly say that I'm fairly good at this point by now, but it's all been very hard-earned. Nobody teaches this stuff, unfortunately.

Matt: 54:20 This is the hole I'm trying to fill, absolutely. Thank you so much, Mark. That was amazing. I've got both of those links. I'll put them in the notes below. Thanks so much for your time.

Mark: 54:32 Sure.