Finite difference methods to approximate PDEs this week
\nNext week we will continue this and do a little bit more complicated stuff.
\nHomework is due right before spring break.
\nExam will be any four hours during the weekend, questions will be a little different.
\nThe first thing is the ODEs, the ordinary differential equations.
\nThere is an entire class about this in every program, as long as you do some kind of engineering science you need to take Calc 3, which covers these.
\nThe reason they’re called ordinary is because they’re about finding a function \\(f(x)\\).
\nThe solution of the equation would be a function, expressed as derivatives of this function.
\nA fake example would be \\(2f''(x) + 3xf'(x) + x^2 + 2 = 0\\)
\nsubject to \\(f(0) = 3\\)
\njust to see how it looks.
\nNormally you spend a lot of time solving first-order, the most complicated one. You learn how to solve equations with constant coefficients, and things that can be reduced by transformations to having constant coefficients. Then you make a polynomial that solves the PDE.
\nThis is Calc 4, partial differential equations. You would only do this if your area requires it.
\nNot necessarily all of us have done this.
\nI thought this was the hardest, most horrible class. My professor purposely failed people (especially young women) so they would pay him for tutoring.
\nThese will usually be a function of more than one variable, like
\n$$\nf(t, x)\n$$
\nThe second variable is usually time, but it doesn’t have to be. x can be multi-dimensional also. Let’s say it’s a line. A practical application would be the heat equation. You have a metal rod and you apply heat to one end, and you look at the distribution of heat over time, f will measure the heat, t will measure time, x will measure location. You could also have a metal plate, and then x is multidimensional.
\n[Embedded figure omitted from the text export.]
\nSo \\(f(t, x)\\) is what we’re trying to find.
\nWe’re solving in the time \\(t \\in (0, \\infty)\\)
\nMost of the time, these problems have the initial condition.
\nThe equations that we will deal with will have a terminal boundary condition, ending at T. You know what the option value is at T.
\nSome equations will start at 0, but you can change between boundary condition 0 and T with change of variables. If you do the \\(\\tau = T - t\\), then it will go from terminal to initial.
\nThe x we’re talking about is \\(x \\in \\mathbb{R}\\), aka one dimensional, but in general \\(x \\in \\mathbb{R}^n\\).
\nHere is a practical example of this. I worked a former Stevens professor some years ago on NYHOPS, New York Harbor Ovserving and Predicting System. Something solves a bunch of PDEs to do forecasts of things like salinity, etc. for the next 48 hours, on a revolving 6 hour period. Actually they just calculate salinity, and it turns out that water speed is driven by salinity.
\nSo how does he do it?
\nHe uses some other equation that is used for viscosity of water, but we don’t need to worry about it.
\nBecause this \\(f(t, x)\\) has two variables, the equations will involve multiple derivatives, joint derivatives.
\nLEt’s say we have a Linear PDE, which we will define as \\(u(t, x)\\). We will use u for our own purposes. This will only have second order derivatives
\n$$\na \\frac{\\partial^2 u}{\\partial t^2} + b \\frac{\\partial^2 u}{\\partial t \\partial x} + c \\frac{\\partial^2 u}{\\partial x^2} + d\\frac{\\partial u}{\\partial t} + e \\frac{\\partial u}{\\partial x} + fu + j\n$$
\nThis is the most general form of a linear PDE. Although they don’t have to be constant, for our definitions they need to be constant. The behavior of our solution, it turns out, is only governed by the second order derivatives. So we only need to worry about a, b, and c.
\nWe will make a polynomial in α and β, where these represent the derivative with respect to t and x, respectively. Then,
\n$$\nP(\\alpha, \\beta) = a\\alpha^2 + b\\alpha \\beta + c \\beta^2 + d\\alpha + e\\beta + ct\n$$
\nAnd again we only worry about a, b, and c.
\nTo reiterate: The nature of the PDE is determined by the properties of these 2nd order terms.
\nSpecifically, we can look at these terms
\n\\(a \\alpha^2 + b \\alpha \\beta + c \\beta^2\\) and divide by \\(\\beta^2\\). THis gives us
\n$$\na(\\frac{\\alpha}{\\beta})^2 + b \\frac{\\alpha}{\\beta} + c\n$$
\nAnd you can see this is a quadratic polynomial. It’s very well-studied and easy to solve. To solve for the determinant, it is
\n$$\n\\Delta = b^2 - 4ac\n$$
\nWe have different behavior based on this Δ.
\nThere are three types of major equations.
\nThe equation has no real solutions, only complex conjugate solutions. These lead specifically to elliptic equations. One example of this that you would have done if you covered PDEs is the Laplace equation.
\nThe Laplace equation looks like
\n$$\n\\frac{\\partial^2 u}{\\partial t^2} + \\frac{\\partial^2 u}{\\partial x^2} = 0\n$$
\nThis is the simplest equation that is elliptic. It appears in thermodynamics. If you’re dealing with gases, behavior in physics, you will see this.
\nThis is called a hyperbolic equation. These terms (by the way) come from the behavior of the solution.
\nWhen you solve a PDE like this, if you don’t have boundary conditions, you get a function which conducts a field. You can look at the derivative of a function and it will tell you where the function goes. You have to tie it down to get one particular surface. The magnetic field is one such function.
\nThe minimum is (this is trivial)
\n$$\n\\frac{\\partial^2 u}{\\partial t^2} - \\frac{partial^2 u}{\\partial x^2} = 0\n$$
\nAnd if you pay attention to where a and c are, you can see where the positive and negative come form.
\nThis is called the wave equation, the jumprope equation which vibrates, useful for earthquakes. The way they do it is by solving an equation of this time, (although with more terms), and the constants are determined by the structure of the earth.
\nThis is called the parabolic PDE. The diffusion equation, if you’ve heard of diffusion tensor imaging, or something like that, uses these kinds of equations. You can see the image of the brain, but it’s not even correct. The brain is made of grey matter, the neurons, and the white matter, which are the axons that connect them. But nobody can cut open your brain to look at the structure. But the white matter decomposes immediately. Nobody knows how they are connected, so you have to put them in the MRI and see how the water molecules move. These are very intrusive, so you can’t do it for too long, then you have to make connections. You can’t see the cables, you can only see the regions. So someone says to take the PDE from the 1970s where they consider the axons are pipes that they heat up.
\nThe heat equation is the most classical example:\n$$\n\\frac{\\partial u}{\\partial t} - \\frac{\\partial^2 u}{\\partial x^2} = 0\n$$
\nYou have two terms that are missing, that’s the only way to get this equation. If you do this with parameters, then you get a square, which will become transformed into first-order aka not interesting. The only way to get second order you have to keep only one term. And in fact you can flip the t and x here and it’s not a big deal.
\nAll finance of any kind uses this equation.
\nBecause this is basically what you get when you have Markov processes. And in general diffusion of particles involves this.
\nIn general, PDEs are solved for \\(t \\in (0, \\infty) \\times x \\in \\mathbb{R}\\). So how does this space look?\n[Embedded figure omitted from the text export.]
\nBut in finance, we are bounded by T:
\nIf I don’t specify the boundary condition, the solution is floating, it can be infinite number of curves. I have to tie it down with the boundary T. And there’s another boundary at x = 0. You tie it down from three different places, t > 0, t < T, and x > 0. And I don’t care about anything other than when t = 0, because that’s where I am now.
\n[Embedded figure omitted from the text export.]
\nIn a nutshell, to solve this, we create a domain, and we make a grid on this domain. In order for this grid to fit, we have to make squares. They solve the equation on each tiny square. Then they propagate the solution (I will show you how soon). This is nothing special, it is general and it applies to any PDE whatsoever.
\nFirst: what is the PDE?
\nLet’s call \\(V(t, S)\\) for the value of an option at time t with asset price S.
\nWe know that if the asset follows risk-neutral geometric Brownian motion, then V solves the following PDE:
\n$$\n\\frac{\\partial V}{\\partial t} + rS\\frac{\\partial V}{\\partial S} + \\frac{1}{2} \\sigma^2 S^2 \\frac{\\partial^2 V}{\\partial S^2} - rV = 0\n$$
\nAnd this is the Black-Scholes PDE (technically, it’s Merton’s because he’s the one that did the PDE.
\nHowever this does not look like the heat equation, because the coefficients are not constant.
\nBut you can make it constant through the use of transformation:\n\\(S = e^x\\), which is the same as \\(x = \\ln S\\)
\nYou can make the t go to T to solve the initial value as well, but this is a different way.
\nThen we can call
\n$$\nV(S, t) = V(e^x, t) = u(x, t)\n$$
\n$$\n\\frac{\\partial V}{\\partial t}(t, s) = \\frac{\\partial u}{\\partial t}(t, x)\n$$
\nSince t doesn’t do anything, this is easy.
\nBut what is
\n\\(\\frac{\\partial V}{\\partial S}\\) in terms of x? I have to substitute two derivatives.
\nBut you can do the chain rule to solve for dvdS
\n$$\n\\frac{\\partial u}{\\partial t} + (r - \\frac{\\sigma^2}{2}) \\frac{\\partial u}{\\partial x} + \\frac{1}{2} \\sigma^2 \\frac{\\partial^2 u}{\\partial x^2} - ru = 0\n$$
\nAnd we’ll call r - σ^2/2 = μ for convenience.
\nBecause this is constant coefficients, you can actually make this into a heat equation which is easy to solve. Merton solves it in a horrible way which is not like that. And you can reduce it into Black-Scholes.
\nBut if there is no analytical solution, how do I approximate this solution? For Black-Scholes, we already have the analytical solution. We learn it as an example here, and once we learn the principle we can apply the methods for problems where there are only numerical solutions. NYHOPS does not have an analytical solution.
\nThere’s two different ways of solving this. There is the explicit and implicit way. They are both finite difference methods. There is another way which is more appropriate to NYHOPS, but we will not cover that.
\nFirst, we take our domain.
\n$$\nt \\in [0, T)\n$$
\nx is different because we did this logarithm.
\n$$\nx = \\ln S = (-\\infty, \\infty)\n$$
\nWe need to discretize this domain. A square grid is the easiest, simplest way to do it. Most PDEs are solved in this way.
\nWe substitute the derivative with the finite difference at each point on the grid. Then we can solve based on where the asset price is. In the explicit method, we use the boundary, and you move from there to the direction you want to go. And you do so explicitly (Will explain soon what that means). Depending on how many terms you have, you need t terms to get one term, in this case 3.
\n[Embedded figure omitted from the text export.]
\nThe difference in this equation, is that you won’t be able to explain each point in terms of one. You have to solve equations in general. Explicit describes the points explicitly in terms of previous points. For implicit, the value is expressed by the value implied by ALL the values on the previous points.
\nGenerally, explicit is easier, but it may not converge. Implicit always converges.
\nBTW, NYHOPS works with curves and boundaries, which is called the finite element method. When you propagate these, you get inconsistencies, so if you have to keep propagating until you get convergence. You solve for the vertices of the cube. Now you have nine points available.
\nThis is the actual math for doing this.
\nWe need to discretize t and x.
\n$$\n\\Delta t = \\frac{T}{n}\n$$
\nvery simple.
\nΔx can be anything, with smaller being better. Is there an ideal relationship between Δx and Δt? There is, but we worry about that later.
\n$$\nt = (0, \\Delta t, 2 \\Delta t, \\ldots, n \\Delta t)\n$$
\n$$\nx = (-N \\Delta x, (-N + 1) \\Delta X, \\ldots, 0, \\Delta x, \\ldots, N \\Delta x)\n$$
\nSo we have
\n2N + 1 points in x, and n + 1 points in t.
\nt thing doesn’t matter, x is crucial.
\nAnd we will notate the value of the function u as
\n$$\nt_i = i\\Delta t\n$$
\n$$\nx_i = j\\Delta x - N\n$$
\n$$\nu(t_i, x_j) = u_{ij}\n$$
\nWe need three derivatives
\nThis one is easy. The big difference is in how I make my derivatives.
\nThey are calculated directly from these three points:
\n[Embedded figure omitted from the text export.]
\nHere are our equations:
\n$$\n\\frac{\\partial u}{\\partial t} = \\frac{u_{i+1,j}-u_{ij}}{\\Delta t}\n$$
\n$$\n\\frac{\\partial u}{\\partial x} = \\frac{u_{i+1, j+1} - u_{i+1, j-1}}{2\\Delta x}\n$$
\n$$\n\\frac{\\partial^2 u}{\\partial x^2} = \\frac{u_{i+1,j+1}-2u_{i+1,j}+u_{i+1,j-1}}{\\Delta x^2}\n$$
\nTo get it, you need those three points.
\nAll of these things are now substituted in our PDE.
\n$$\n\\frac{\\partial u}{\\partial t} + \\mu \\frac{\\partial u}{\\partial x} + \\frac{1}{2} \\sigma^2 \\frac{\\partial^2 u}{\\partial x^2} - ru = 0\n$$
\nJust the points with some constants like μ and σ.
\nNow you can determine those constants in terms of the other terms:
\n$$\nu_{ij} = \\Delta t(\\frac{\\sigma^2}{2\\Delta x^2} + \\frac{\\mu}{2\\Delta x}) \\mu_{i+1,j+1} + (\\text{a term})u_{i+1,j} + \\text{another term}u_{i+1,j-1}\n$$
\nIf we take those coefficient terms to be \\(P_u\\), \\(P_m\\), and \\(P_d\\), respectively, the probability of going up, staying the same, or going down, then it turns into a trinomial.
\nAnd it’s almost the same as the formula for the trinomial tree! The only difference is that explicit finite difference has \\(\\frac{1}{1+r\\Delta t}\\) inside the formulas, whereas the trinomial uses \\(\\frac{1}{e^{r\\Delta t}}\\) aka discrete vs continuous. And of course as N increases we approach continuous.
\nIn order for this thing to converge, we must have \\(\\Delta x \\geq \\sigma \\sqrt{3\\Delta t}\\) which is the same condition as the trinomial tree. You can make Δx small, but not too small. Conceptually, Δx tells me how many points I have to solve for, and Δt tells me the number of steps I have to go through the tree. Because we have 3 to 1, we must have N > n. We’re working around the grid, and N tells me how many points I have. And you can see this visually, that you will get stuck if you have too many Δx compared to Δt:\n[Embedded figure omitted from the text export.]
\nIf I pick \\(\\Delta x = \\sigma \\sqrt{3 \\Delta t}\\).
\nIf you calculate Δ you need two points in the origin, Γ you need three points. Sometimes that is useful.
\nFundamentally, if you understand the explicit, it’s similar.
\nKeep in mind the points. We’ll be using FOUR POINTS for the three derivatives.
\n[Embedded figure omitted from the text export.]
\nIt’s hard to remember formulas. But it’s easier to remember how to derive them.
\nNow I’m going to plug them into the PDE, and then solve it. Remember that these coefficients are constants. Then you will get an equation relating these four points
\n$$\nAu_{i,j+1} + Bu_{i,j} + Cu_{i,j-1} = u_{i+1,j}\n$$
\nThe NUMBER ONE MOST IMPORTANT THING IS:
\nA, B, C are the same for all i,j. This is normal because the coefficients of the PDE don’t depend on the location or time, no t or x in it. That makes it simpler to solve!
\nNonetheless, this is still an implicit solution.
\nA, B, C are not probabilities anymore.
\nWe previously generalized American options by calculating the expected value of the future value of the option given that you are at that point. So you could take that point and compare what happens at exercise and store it. However, this doesn’t work, so it loses the interpretation of expected value.
\nThere will be an exercise about this.
\nWe have three unknowns and one equation, so we have to solve a lot of equations. We have 2N + 1 points. Because we are writing one equation for each set of three points, we will be lacking two equations, so we have 2N - 2 equations. So the system built here CANNOT be solved.
\nTherefore we need to come up with two more equations.
\nThe two extra equations are coming from boundary conditions. Remember, N is supposed to be very large. And our underlying is the log of the stock, these boundaries relate to very high and very low stock value.\nIt depends, because for a call, when a stock rises in value, the call becomes very valuable, for a put, it becomes worthless.
\nFor Put, when \\(S \\uparrow \\infty\\), then value is small. Under our property of log, it can’t be 0, although obviously we think it is.
\n$$\n\\frac{\\partial V}{\\partial S} = 0\n$$
\nbecause it’s not going to change at all.
\nAnd then when \\(S \\downarrow 0\\)
\n$$\n\\frac{\\partial V}{\\partial S} = -1\n$$
\nThis comes from the fact that K is a constant and derivative of S by itself is 1.
\nIf we consider the topmost \\(i,N\\). The difference
\n$$\n\\frac{u_{i,N} - u_{i,N-1}}{e^{N\\Delta x + x_0} - e^{(N-1)\\Delta x + x_0}} = 0\n$$
\nWe should not be using u, but the corresponding Vs. We need to use the unknowns that we have, not other unknowns. So we can’t use dV/dS, we have to use the corresponding du/dS.
\nAnd we can get an extra equation from
\n$$\nu_{i,N} - u_{i,N-1} = 0\n$$
\nFor the bottom put, it’s
\n$$\n\\frac{u_{i,-N} - u_{i, -N+1}}{e^{(-N+1)\\Delta x + x_0} - e^{-N\\Delta x + x_0}}\n$$
\nand that becomes
\n$$\nu_{i, -N} - u_{i, -N+1} = \\lambda_D\n$$
\nwhere λ is some number.
\n$$\nu_{i,N} - u_{i,N-1} = \\lambda_U\n$$
\n$$\nu_{i, -N} - u_{i, -N+1} = \\lambda_D\n$$
\nAnd these are two extra equations that we can use to solve.
\nBut this matrix is HUGE, 2N+1 by 2N+1, will cripple your computer. We deal with imbeciles that solve by exhaustive search, that’s how LLMs work. If you take these matrix 201x201 and ask R to solve it, it’s very fast. Then 1000x1000 is a little slower, but it can still do it. However, you can solve this with your brain, by derivation.
\nIt is three formulas, which are easy to implement!
\nSince basically only the diagonal is populated, we are wasting our time doing typical matrix solving.
\nThe idea is kinda cool. We know the matrix is solvable, because it’s invertible. We know that because the determinant exists and is not 0, easy to show.
\nThere is a general method called the Jacobian, where you come up with some random numbers and plug them in. But there are two other methods to solve, the 5th grade ones: substitution and elimination.
\nAnd you can solve it in this way.
\nThe first equation is
\n$$\nu_N u_{N-1}\n$$
\nYou can express the latter in terms of the former..
\nThen you can express \\(u_{N-2}\\) in the same way, and keep bootstrapping to make it all functions of \\(u_N\\).
\nThen you go to the end and get
\n\\(u_{N-1}\\) and \\(u_{-N}\\) and get \\(u_N\\) from there, and then get everything else from there.
\nThis is called in the book solving a tridiagonal system. Because you have three main diagonals, this is possible.
\nTechnically you can solve this in general.
\nAnd this is in the book too.
\nWe will assume matrix A\n[Embedded figure omitted from the text export.]
\nYou’ll see that this works with the numbers being all different, although it’s easier in our case with the numbers being the same.
\nTo solve this, we go through the motion:
\n$$\na_{11} x_1 + a_{12} x_2 = y_1\n$$
\n$$\nx_1 = \\frac{1}{a_{11}} y_1 - \\frac{a_{12}}{a_{11}}x_2\n$$
\nLet’s look at this structure. We have a number - another number times x_2. We’ll call those numbers C_1 and D_1.
\n$$\nx_1 = C_1 + D_1 x_2\n$$
\nThen
\n$$\nx_2 = \\frac{y_2 - a_{21} C_1}{a_{21}D_1 + a_{22}} - \\frac{a_{23}}{a_{21} D_1 + a_{22}} x_3\n$$
\nAnd you can see it’s the same type of expression as x_1.
\nI came up with this on my own, the general idea of the derivation.
\nIn general, each step i follows
\n$$\nx_i = C_i + D_i x_{i+1}\n$$
\nwhere
\n$$\nC_i = \\frac{y_i - a_{i, i -1} C_{i-1}}{a_{i, i-1} D_{i - 1} + a_{i,i}}\n$$
\nand
\n$$\nD_i = -\\frac{a_{i, i+1}}{a_{i, i-1} D_{i-1} + a_{ii}}\n$$
\nIn general, \\(y_i\\) is
\n$$\na_{i+1, i}x_i + a_{i+1,i+1} x_{i+1} + a_{i+1,i+2} x_{i+2} = y_{i+1}\n$$
\nwe can substitute the same C and D terms as we had done in 2, and turns out to be the exact same terms.
\nThat means it’s really easy to program!
\nThe only thing is, the end gets treated a little differently.
\nThe last equation:
\n$$\na_{n, n-1} x_{n-1} + a_{nn} x_n = y_n\n$$
\nWe take that x value, and we say it’s equal to
\n$$\nx_{n-1} = C_{n-1} + D_{n-1} x_n\n$$
\nIt makes it easy to express in the future, because then you can move them all to the end. Once you plug that in backwards, you’re going to get x_n, so
\n$$\nx_n = \\frac{y_n - a_{n,n-1} C_{n-1}}{a_{n,n-1} D_{n-1} + a_{nn}}\n$$
\nAND THEN I’M DONE!
","contentMarkdown":"## Plan for the next weeks\n\nFinite difference methods to approximate PDEs this week\n\nNext week we will continue this and do a little bit more complicated stuff.\n\nHomework is due right before spring break.\n\nExam will be any four hours during the weekend, questions will be a little different.\n\n## Differential Equations\n\nThe first thing is the ODEs, the ordinary differential equations.\n\nThere is an entire class about this in every program, as long as you do some kind of engineering science you need to take Calc 3, which covers these.\n\nThe reason they’re called ordinary is because they’re about finding a function \\\\(f(x)\\\\).\n\nThe solution of the equation would be a function, expressed as derivatives of this function.\n\nA fake example would be \\\\(2f''(x) + 3xf'(x) + x^2 + 2 = 0\\\\)\n\nsubject to \\\\(f(0) = 3\\\\)\n\njust to see how it looks.\n\nNormally you spend a lot of time solving first-order, the most complicated one. You learn how to solve equations with constant coefficients, and things that can be reduced by transformations to having constant coefficients. Then you make a polynomial that solves the PDE.\n\n## PDEs\n\nThis is Calc 4, partial differential equations. You would only do this if your area requires it.\n\nNot necessarily all of us have done this.\n\nI thought this was the hardest, most horrible class. My professor purposely failed people (especially young women) so they would pay him for tutoring.\n\nThese will usually be a function of more than one variable, like\n\n$$\nf(t, x)\n$$\n\nThe second variable is usually time, but it doesn’t have to be. x can be multi-dimensional also. Let’s say it’s a line. A practical application would be the heat equation. You have a metal rod and you apply heat to one end, and you look at the distribution of heat over time, f will measure the heat, t will measure time, x will measure location. You could also have a metal plate, and then x is multidimensional.\n\n[Embedded figure omitted from the text export.]\n\nSo \\\\(f(t, x)\\\\) is what we’re trying to find.\n\nWe’re solving in the time \\\\(t \\\\in (0, \\\\infty)\\\\)\n\nMost of the time, these problems have the initial condition.\n\nThe equations that we will deal with will have a **terminal boundary condition**, ending at T. You know what the option value is at T.\n\nSome equations will start at 0, but you can change between boundary condition 0 and T with change of variables. If you do the \\\\(\\\\tau = T - t\\\\), then it will go from terminal to initial.\n\nThe x we’re talking about is \\\\(x \\\\in \\\\mathbb{R}\\\\), aka one dimensional, but in general \\\\(x \\\\in \\\\mathbb{R}^n\\\\).\n\n## NYHOPS\n\nHere is a practical example of this. I worked a former Stevens professor some years ago on NYHOPS, New York Harbor Ovserving and Predicting System. Something solves a bunch of PDEs to do forecasts of things like salinity, etc. for the next 48 hours, on a revolving 6 hour period. Actually they just calculate salinity, and it turns out that water speed is driven by salinity.\n\nSo how does he do it?\n\nHe uses some other equation that is used for viscosity of water, but we don’t need to worry about it.\n\nBecause this \\\\(f(t, x)\\\\) has two variables, the equations will involve multiple derivatives, **joint derivatives**.\n\nLEt’s say we have a **Linear PDE**, which we will define as \\\\(u(t, x)\\\\). We will use u for our own purposes. This will only have second order derivatives\n\n$$\na \\\\frac{\\\\partial^2 u}{\\\\partial t^2} + b \\\\frac{\\\\partial^2 u}{\\\\partial t \\\\partial x} + c \\\\frac{\\\\partial^2 u}{\\\\partial x^2} + d\\\\frac{\\\\partial u}{\\\\partial t} + e \\\\frac{\\\\partial u}{\\\\partial x} + fu + j\n$$\n\nThis is the most general form of a linear PDE. Although they don’t have to be constant, for our definitions they need to be constant. The behavior of our solution, it turns out, is only governed by the second order derivatives. So we only need to worry about a, b, and c.\n\nWe will make a polynomial in α and β, where these represent the derivative with respect to t and x, respectively. Then,\n\n$$\nP(\\\\alpha, \\\\beta) = a\\\\alpha^2 + b\\\\alpha \\\\beta + c \\\\beta^2 + d\\\\alpha + e\\\\beta + ct\n$$\n\nAnd again we only worry about a, b, and c.\n\nTo reiterate: **The nature of the PDE is determined by the properties of these 2nd order terms.**\n\nSpecifically, we can look at these terms\n\n\\\\(a \\\\alpha^2 + b \\\\alpha \\\\beta + c \\\\beta^2\\\\) and divide by \\\\(\\\\beta^2\\\\). THis gives us\n\n$$\na(\\\\frac{\\\\alpha}{\\\\beta})^2 + b \\\\frac{\\\\alpha}{\\\\beta} + c\n$$\n\nAnd you can see this is a quadratic polynomial. It’s very well-studied and easy to solve. To solve for the determinant, it is\n\n$$\n\\\\Delta = b^2 - 4ac\n$$\n\nWe have different behavior based on this Δ.\n\nThere are three types of major equations.\n\n### Δ \\< 0\n\nThe equation has no real solutions, only complex conjugate solutions. These lead specifically to **elliptic equations**. One example of this that you would have done if you covered PDEs is the **Laplace equation**.\n\nThe Laplace equation looks like\n\n$$\n\\\\frac{\\\\partial^2 u}{\\\\partial t^2} + \\\\frac{\\\\partial^2 u}{\\\\partial x^2} = 0\n$$\n\nThis is the simplest equation that is elliptic. It appears in thermodynamics. If you’re dealing with gases, behavior in physics, you will see this.\n\n### Δ \\> 0\n\nThis is called a **hyperbolic equation**. These terms (by the way) come from the behavior of the solution.\n\nWhen you solve a PDE like this, if you don’t have boundary conditions, you get a function which conducts a field. You can look at the derivative of a function and it will tell you where the function goes. You have to tie it down to get one particular surface. The magnetic field is one such function.\n\nThe minimum is (this is trivial)\n\n$$\n\\\\frac{\\\\partial^2 u}{\\\\partial t^2} - \\\\frac{partial^2 u}{\\\\partial x^2} = 0\n$$\n\nAnd if you pay attention to where a and c are, you can see where the positive and negative come form.\n\nThis is called the **wave equation**, the jumprope equation which vibrates, useful for earthquakes. The way they do it is by solving an equation of this time, (although with more terms), and the constants are determined by the structure of the earth.\n\n### Δ \\= 0\n\nThis is called the **parabolic PDE**. The **diffusion equation**, if you’ve heard of diffusion tensor imaging, or something like that, uses these kinds of equations. You can see the image of the brain, but it’s not even correct. The brain is made of grey matter, the neurons, and the white matter, which are the axons that connect them. But nobody can cut open your brain to look at the structure. But the white matter decomposes immediately. Nobody knows how they are connected, so you have to put them in the MRI and see how the water molecules move. These are very intrusive, so you can’t do it for too long, then you have to make connections. You can’t see the cables, you can only see the regions. So someone says to take the PDE from the 1970s where they consider the axons are pipes that they heat up.\n\nThe **heat equation** is the most classical example:\n$$\n\\\\frac{\\\\partial u}{\\\\partial t} - \\\\frac{\\\\partial^2 u}{\\\\partial x^2} = 0\n$$\n\nYou have two terms that are missing, that’s the only way to get this equation. If you do this with parameters, then you get a square, which will become transformed into first-order aka not interesting. The only way to get second order you have to keep only one term. And in fact you can flip the t and x here and it’s not a big deal.\n\n**All finance of any kind uses this equation**.\n\nBecause this is basically what you get when you have Markov processes. And in general diffusion of particles involves this.\n\n## Boundary Conditions\n\nIn general, PDEs are solved for \\\\(t \\\\in (0, \\\\infty) \\\\times x \\\\in \\\\mathbb{R}\\\\). So how does this space look?\n[Embedded figure omitted from the text export.]\n\nBut in finance, we are bounded by T:\n\nIf I don’t specify the boundary condition, the solution is floating, it can be infinite number of curves. I have to tie it down with the boundary T. And there’s another boundary at x \\= 0\\. You tie it down from three different places, t \\> 0, t \\< T, and x \\> 0\\. And I don’t care about anything other than when t \\= 0, because that’s where I am now.\n\n[Embedded figure omitted from the text export.]\n\nIn a nutshell, to solve this, we create a domain, and we make a grid on this domain. In order for this grid to fit, we have to make squares. They solve the equation on each tiny square. Then they propagate the solution (I will show you how soon). This is nothing special, it is general and it applies to any PDE whatsoever.\n\n## Methodology\n\nFirst: what is the PDE?\n\nLet’s call \\\\(V(t, S)\\\\) for the value of an option at time t with asset price S.\n\nWe know that if the asset follows risk-neutral geometric Brownian motion, then V solves the following PDE:\n\n$$\n\\\\frac{\\\\partial V}{\\\\partial t} + rS\\\\frac{\\\\partial V}{\\\\partial S} + \\\\frac{1}{2} \\\\sigma^2 S^2 \\\\frac{\\\\partial^2 V}{\\\\partial S^2} - rV = 0\n$$\n\nAnd this is the Black-Scholes PDE (technically, it’s Merton’s because he’s the one that did the PDE.\n\nHowever this does not look like the heat equation, because the coefficients are not constant.\n\nBut you can make it constant through the use of transformation:\n\\\\(S = e^x\\\\), which is the same as \\\\(x = \\\\ln S\\\\)\n\nYou can make the t go to T to solve the initial value as well, but this is a different way.\n\nThen we can call\n\n$$\nV(S, t) = V(e^x, t) = u(x, t)\n$$\n\n$$\n\\\\frac{\\\\partial V}{\\\\partial t}(t, s) = \\\\frac{\\\\partial u}{\\\\partial t}(t, x)\n$$\n\nSince t doesn’t do anything, this is easy.\n\nBut what is\n\n\\\\(\\\\frac{\\\\partial V}{\\\\partial S}\\\\) in terms of x? I have to substitute two derivatives.\n\nBut you can do the chain rule to solve for dvdS\n\n$$\n\\\\frac{\\\\partial u}{\\\\partial t} + (r - \\\\frac{\\\\sigma^2}{2}) \\\\frac{\\\\partial u}{\\\\partial x} + \\\\frac{1}{2} \\\\sigma^2 \\\\frac{\\\\partial^2 u}{\\\\partial x^2} - ru = 0\n$$\n\nAnd we’ll call r \\- σ^2/2 \\= μ for convenience.\n\nBecause this is constant coefficients, you can actually make this into a heat equation which is easy to solve. Merton solves it in a horrible way which is not like that. And you can reduce it into Black-Scholes.\n\nBut if there is no analytical solution, how do I approximate this solution? For Black-Scholes, we already have the analytical solution. We learn it as an example here, and once we learn the principle we can apply the methods for problems where there are only numerical solutions. NYHOPS does not have an analytical solution.\n\nThere’s two different ways of solving this. There is the **explicit** and **implicit** way. They are both **finite difference methods**. There is another way which is more appropriate to NYHOPS, but we will not cover that.\n\n## Explicit Finite Difference\n\nFirst, we take our domain.\n\n$$\nt \\\\in [0, T)\n$$\n\nx is different because we did this logarithm.\n\n$$\nx = \\\\ln S = (-\\\\infty, \\\\infty)\n$$\n\nWe need to discretize this domain. A square grid is the easiest, simplest way to do it. Most PDEs are solved in this way.\n\nWe substitute the derivative with the finite difference at each point on the grid. Then we can solve based on where the asset price is. In the explicit method, we use the boundary, and you move from there to the direction you want to go. And you do so explicitly (Will explain soon what that means). Depending on how many terms you have, you need t terms to get one term, in this case 3\\.\n\n[Embedded figure omitted from the text export.]\n\n## Implicit\n\nThe difference in this equation, is that you won’t be able to explain each point in terms of one. You have to solve equations in general. Explicit describes the points explicitly in terms of previous points. For implicit, the value is expressed by the value implied by ALL the values on the previous points.\n\n## Differences\n\nGenerally, explicit is easier, but it may not converge. Implicit always converges.\n\nBTW, NYHOPS works with curves and boundaries, which is called the finite element method. When you propagate these, you get inconsistencies, so if you have to keep propagating until you get convergence. You solve for the vertices of the cube. Now you have nine points available.\n\n## How\n\nThis is the actual math for doing this.\n\n### Discretize the Domain\n\nWe need to discretize t and x.\n\n$$\n\\\\Delta t = \\\\frac{T}{n}\n$$\n\nvery simple.\n\nΔx can be anything, with smaller being better. Is there an ideal relationship between Δx and Δt? There is, but we worry about that later.\n\n$$\nt = (0, \\\\Delta t, 2 \\\\Delta t, \\\\ldots, n \\\\Delta t)\n$$\n\n$$\nx = (-N \\\\Delta x, (-N + 1) \\\\Delta X, \\\\ldots, 0, \\\\Delta x, \\\\ldots, N \\\\Delta x)\n$$\n\nSo we have\n\n2N \\+ 1 points in x, and n \\+ 1 points in t.\n\nt thing doesn’t matter, x is crucial.\n\nAnd we will notate the value of the function u as\n\n$$\nt_i = i\\\\Delta t\n$$\n\n$$\nx_i = j\\\\Delta x - N\n$$\n\n$$\nu(t_i, x_j) = u_{ij}\n$$\n\nWe need three derivatives\n\n\n### Explicit\n\nThis one is easy. The big difference is in how I make my derivatives.\n\nThey are calculated directly from these three points:\n\n[Embedded figure omitted from the text export.]\n\nHere are our equations:\n\n$$\n\\\\frac{\\\\partial u}{\\\\partial t} = \\\\frac{u_{i+1,j}-u_{ij}}{\\\\Delta t}\n$$\n\n$$\n\\\\frac{\\\\partial u}{\\\\partial x} = \\\\frac{u_{i+1, j+1} - u_{i+1, j-1}}{2\\\\Delta x}\n$$\n\n$$\n\\\\frac{\\\\partial^2 u}{\\\\partial x^2} = \\\\frac{u_{i+1,j+1}-2u_{i+1,j}+u_{i+1,j-1}}{\\\\Delta x^2}\n$$\n\nTo get it, you need those three points.\n\nAll of these things are now substituted in our PDE.\n\n$$\n\\\\frac{\\\\partial u}{\\\\partial t} + \\\\mu \\\\frac{\\\\partial u}{\\\\partial x} + \\\\frac{1}{2} \\\\sigma^2 \\\\frac{\\\\partial^2 u}{\\\\partial x^2} - ru = 0\n$$\n\nJust the points with some constants like μ and σ.\n\nNow you can determine those constants in terms of the other terms:\n\n$$\nu_{ij} = \\\\Delta t(\\\\frac{\\\\sigma^2}{2\\\\Delta x^2} + \\\\frac{\\\\mu}{2\\\\Delta x}) \\\\mu_{i+1,j+1} + (\\\\text{a term})u_{i+1,j} + \\\\text{another term}u_{i+1,j-1}\n$$\n\nIf we take those coefficient terms to be \\\\(P_u\\\\), \\\\(P_m\\\\), and \\\\(P_d\\\\), respectively, the probability of going up, staying the same, or going down, then it turns into a trinomial.\n\nAnd it’s almost the same as the formula for the trinomial tree\\! The only difference is that explicit finite difference has \\\\(\\\\frac{1}{1+r\\\\Delta t}\\\\) inside the formulas, whereas the trinomial uses \\\\(\\\\frac{1}{e^{r\\\\Delta t}}\\\\) aka discrete vs continuous. And of course as N increases we approach continuous.\n\n## Stability and Convergence\n\nIn order for this thing to converge, we must have \\\\(\\\\Delta x \\\\geq \\\\sigma \\\\sqrt{3\\\\Delta t}\\\\) which is the same condition as the trinomial tree. You can make Δx small, but not too small. Conceptually, Δx tells me how many points I have to solve for, and Δt tells me the number of steps I have to go through the tree. Because we have 3 to 1, we must have N \\> n. We’re working around the grid, and N tells me how many points I have. And you can see this visually, that you will get stuck if you have too many Δx compared to Δt:\n[Embedded figure omitted from the text export.]\n\nIf I pick \\\\(\\\\Delta x = \\\\sigma \\\\sqrt{3 \\\\Delta t}\\\\).\n\nIf you calculate Δ you need two points in the origin, Γ you need three points. Sometimes that is useful.\n\n## Implicit Scheme\n\nFundamentally, if you understand the explicit, it’s similar.\n\nKeep in mind the points. We’ll be using FOUR POINTS for the three derivatives.\n\n[Embedded figure omitted from the text export.]\n\nIt’s hard to remember formulas. But it’s easier to remember how to derive them.\n\nNow I’m going to plug them into the PDE, and then solve it. Remember that these coefficients are constants. Then you will get an equation relating these four points\n\n$$\nAu_{i,j+1} + Bu_{i,j} + Cu_{i,j-1} = u_{i+1,j}\n$$\n\nThe NUMBER ONE MOST IMPORTANT THING IS:\n\n**A, B, C are the same for all i,j**. This is normal because the coefficients of the PDE don’t depend on the location or time, no t or x in it. That makes it simpler to solve\\!\n\nNonetheless, this is still an implicit solution.\n\n**A, B, C are not probabilities anymore**.\n\nWe previously generalized American options by calculating the expected value of the future value of the option given that you are at that point. So you could take that point and compare what happens at exercise and store it. However, this doesn’t work, so it loses the interpretation of expected value.\n\n**There will be an exercise about this**.\n\nWe have three unknowns and one equation, so we have to solve a lot of equations. We have 2N \\+ 1 points. Because we are writing one equation for each set of three points, we will be lacking two equations, so we have 2N \\- 2 equations. So the system built here CANNOT be solved.\n\nTherefore we need to come up with two more equations.\n\nThe two extra equations are coming from boundary conditions. Remember, N is supposed to be very large. And our underlying is the log of the stock, these boundaries relate to very high and very low stock value.\nIt depends, because for a call, when a stock rises in value, the call becomes very valuable, for a put, it becomes worthless.\n\nFor Put, when \\\\(S \\\\uparrow \\\\infty\\\\), then value is small. Under our property of log, it can’t be 0, although obviously we think it is.\n\n$$\n\\\\frac{\\\\partial V}{\\\\partial S} = 0\n$$\n\nbecause it’s not going to change at all.\n\nAnd then when \\\\(S \\\\downarrow 0\\\\)\n\n$$\n\\\\frac{\\\\partial V}{\\\\partial S} = -1\n$$\n\nThis comes from the fact that K is a constant and derivative of S by itself is 1\\.\n\nIf we consider the topmost \\\\(i,N\\\\). The difference\n\n$$\n\\\\frac{u_{i,N} - u_{i,N-1}}{e^{N\\\\Delta x + x_0} - e^{(N-1)\\\\Delta x + x_0}} = 0\n$$\n\nWe should not be using u, but the corresponding Vs. We need to use the unknowns that we have, not other unknowns. So we can’t use dV/dS, we have to use the corresponding du/dS.\n\nAnd we can get an extra equation from\n\n$$\nu_{i,N} - u_{i,N-1} = 0\n$$\n\nFor the bottom put, it’s\n\n$$\n\\\\frac{u_{i,-N} - u_{i, -N+1}}{e^{(-N+1)\\\\Delta x + x_0} - e^{-N\\\\Delta x + x_0}}\n$$\n\nand that becomes\n\n$$\nu_{i, -N} - u_{i, -N+1} = \\\\lambda_D\n$$\n\nwhere λ is some number.\n\n$$\nu_{i,N} - u_{i,N-1} = \\\\lambda_U\n$$\n\n$$\nu_{i, -N} - u_{i, -N+1} = \\\\lambda_D\n$$\n\nAnd these are two extra equations that we can use to solve.\n\nBut this matrix is HUGE, 2N+1 by 2N+1, will cripple your computer. We deal with imbeciles that solve by exhaustive search, that’s how LLMs work. If you take these matrix 201x201 and ask R to solve it, it’s very fast. Then 1000x1000 is a little slower, but it can still do it. However, you can solve this with your brain, by derivation.\n\nIt is three formulas, which are easy to implement\\!\n\nSince basically only the diagonal is populated, we are wasting our time doing typical matrix solving.\n\nThe idea is kinda cool. We know the matrix is solvable, because it’s invertible. We know that because the determinant exists and is not 0, easy to show.\n\nThere is a general method called the **Jacobian**, where you come up with some random numbers and plug them in. But there are two other methods to solve, the 5th grade ones: substitution and elimination.\n\nAnd you can solve it in this way.\n\nThe first equation is\n\n$$\nu_N u_{N-1}\n$$\n\nYou can express the latter in terms of the former..\n\nThen you can express \\\\(u_{N-2}\\\\) in the same way, and keep bootstrapping to make it all functions of \\\\(u_N\\\\).\n\nThen you go to the end and get\n\n\\\\(u_{N-1}\\\\) and \\\\(u_{-N}\\\\) and get \\\\(u_N\\\\) from there, and then get everything else from there.\n\nThis is called in the book **solving a tridiagonal system**. Because you have three main diagonals, this is possible.\n\nTechnically you can solve this in general.\n\nAnd this is in the book too.\n\nWe will assume matrix A\n[Embedded figure omitted from the text export.]\n\nYou’ll see that this works with the numbers being all different, although it’s easier in our case with the numbers being the same.\n\nTo solve this, we go through the motion:\n\n$$\na_{11} x_1 + a_{12} x_2 = y_1\n$$\n\n$$\nx_1 = \\\\frac{1}{a_{11}} y_1 - \\\\frac{a_{12}}{a_{11}}x_2\n$$\n\nLet’s look at this structure. We have a number \\- another number times x\\_2. We’ll call those numbers C\\_1 and D\\_1.\n\n$$\nx_1 = C_1 + D_1 x_2\n$$\n\nThen\n\n$$\nx_2 = \\\\frac{y_2 - a_{21} C_1}{a_{21}D_1 + a_{22}} - \\\\frac{a_{23}}{a_{21} D_1 + a_{22}} x_3\n$$\n\nAnd you can see it’s the same type of expression as x\\_1.\n\nI came up with this on my own, the general idea of the derivation.\n\nIn general, each step i follows\n\n$$\nx_i = C_i + D_i x_{i+1}\n$$\n\nwhere\n\n$$\nC_i = \\\\frac{y_i - a_{i, i -1} C_{i-1}}{a_{i, i-1} D_{i - 1} + a_{i,i}}\n$$\n\nand\n\n$$\nD_i = -\\\\frac{a_{i, i+1}}{a_{i, i-1} D_{i-1} + a_{ii}}\n$$\n\nIn general, \\\\(y_i\\\\) is\n\n$$\na_{i+1, i}x_i + a_{i+1,i+1} x_{i+1} + a_{i+1,i+2} x_{i+2} = y_{i+1}\n$$\n\nwe can substitute the same C and D terms as we had done in 2, and turns out to be the exact same terms.\n\nThat means it’s really easy to program\\!\n\nThe only thing is, the end gets treated a little differently.\n\nThe last equation:\n\n$$\na_{n, n-1} x_{n-1} + a_{nn} x_n = y_n\n$$\n\nWe take that x value, and we say it’s equal to\n\n$$\nx_{n-1} = C_{n-1} + D_{n-1} x_n\n$$\n\nIt makes it easy to express in the future, because then you can move them all to the end. Once you plug that in backwards, you’re going to get x\\_n, so\n\n$$\nx_n = \\\\frac{y_n - a_{n,n-1} C_{n-1}}{a_{n,n-1} D_{n-1} + a_{nn}}\n$$\n\nAND THEN I’M DONE\\!","dataUrl":"https://sharifhsn.dev/api/posts/computational-methods-week-05.json","date":"2025-02-25","datePublished":"2025-02-25","description":"Finite difference methods to approximate PDEs this week","site":"https://sharifhsn.dev","slug":"computational-methods-week-05","source":"Computational Methods in Quantitative Finance","sourceUrl":null,"tags":["Computational Methods","Differential Equations","PDEs","Finite Differences","Black-Scholes"],"title":"Differential Equations and PDE Methods","url":"https://sharifhsn.dev/blog/computational-methods-week-05/","version":"1","wordCount":3705}