31. Job Search III: Search with Learning#

31.1.1. Model features#

• Infinite horizon dynamic programming with two states and one binary control.

• Bayesian updating to learn the unknown distribution.

31.2.1. The Basic McCall Model#

Recall that, in the baseline model, an unemployed worker is presented in each period with a permanent job offer at wage $W_t$.

At time $t$, our worker either

1. accepts the offer and works permanently at constant wage $W_t$

2. rejects the offer, receives unemployment compensation $c$ and reconsiders next period

The wage sequence $\{W_t\}$ is iid and generated from known density $h$.

The worker aims to maximize the expected discounted sum of earnings $\mathbb{E}\sum _{t=0}^{\mathrm{\infty }}{\beta }^t{y}_t$ The function $V$ satisfies the recursion

The optimal policy has the form $\mathbf{1}\{w\ge \overline{w}\}$, where $\overline{w}$ is a constant depending called the reservation wage.

31.2.2. Offer Distribution Unknown#

Now let’s extend the model by considering the variation presented in [LS18], section 6.6.

The model is as above, apart from the fact that

• the density $h$ is unknown

• the worker learns about $h$ by starting with a prior and updating based on wage offers that he/she observes

The worker knows there are two possible distributions $F$ and $G$ — with densities $f$ and $g$.

At the start of time, “nature” selects $h$ to be either $f$ or $g$ — the wage distribution from which the entire sequence $\{W_t\}$ will be drawn.

This choice is not observed by the worker, who puts prior probability ${\pi }_0$ on $f$ being chosen.

Update rule: worker’s time $t$ estimate of the distribution is ${\pi }_{t}f+(1-{\pi }_t)g$, where ${\pi }_t$ updates via

This last expression follows from Bayes’ rule, which tells us that

The fact that (31.2) is recursive allows us to progress to a recursive solution method.

Letting

we can express the value function for the unemployed worker recursively as follows

Notice that the current guess $\pi$ is a state variable, since it affects the worker’s perception of probabilities for future rewards.

31.2.3. Parameterization#

Following section 6.6 of [LS18], our baseline parameterization will be

• $f$ is $\mathrm{Beta}(1,1)$ scaled (i.e., draws are multiplied by) some factor $w_m$

• $g$ is $\mathrm{Beta}(3,1.2)$ scaled (i.e., draws are multiplied by) the same factor $w_m$

• $\beta =0.95$ and $c=0.6$

With $w_m=2$, the densities $f$ and $g$ have the following shape

using LinearAlgebra, Statistics, SpecialFunctions
using Distributions, LaTeXStrings, Plots, Interpolations, FastGaussQuadrature, NLsolve

w_max = 2
x = range(0, w_max, length = 200)

G = Beta(3, 1.6)
F = Beta(1, 1)
plot(x, pdf.(G, x / w_max) / w_max, label = L"g")
plot!(x, pdf.(F, x / w_max) / w_max, label = L"f")

31.2.4. Looking Forward#

What kind of optimal policy might result from (31.3) and the parameterization specified above?

Intuitively, if we accept at $w_a$ and $w_a\le w_b$, then — all other things being given — we should also accept at $w_b$.

This suggests a policy of accepting whenever $w$ exceeds some threshold value $\overline{w}$.

But $\overline{w}$ should depend on $\pi$ — in fact it should be decreasing in $\pi$ because

• $f$ is a less attractive offer distribution than $g$

• larger $\pi$ means more weight on $f$ and less on $g$

Thus larger $\pi$ depresses the worker’s assessment of her future prospects, and relatively low current offers become more attractive.

Summary: We conjecture that the optimal policy is of the form $\mathbb{1}\{w\ge \overline{w}(\pi )\}$ for some decreasing function $\overline{w}$.

31.4.1. Another Functional Equation#

To begin, note that when $w=\overline{w}(\pi )$, the worker is indifferent between accepting and rejecting.

Hence the two choices on the right-hand side of (31.3) have equal value:

Together, (31.3) and (31.4) give

Combining (31.4) and (31.5), we obtain

Multiplying by $1-\beta$, substituting in ${\pi }^{\prime }=q(w^{\prime },\pi )$ and using $\circ$ for composition of functions yields

Equation (31.6) can be understood as a functional equation, where $\overline{w}$ is the unknown function.

• Let’s call it the reservation wage functional equation (RWFE).

• The solution $\overline{w}$ to the RWFE is the object that we wish to compute.

31.4.2. Solving the RWFE#

To solve the RWFE, we will first show that its solution is the fixed point of a contraction mapping.

To this end, let

• $b[0,1]$ be the bounded real-valued functions on $[0,1]$

• $\Vert \psi \Vert :=\underset{x\in [0,1]}{sup}|\psi (x)|$

Consider the operator $Q$ mapping $\psi \in b[0,1]$ into $Q\psi \in b[0,1]$ via

Comparing (31.6) and (31.7), we see that the set of fixed points of $Q$ exactly coincides with the set of solutions to the RWFE.

• If $Q\overline{w}=\overline{w}$ then $\overline{w}$ solves (31.6) and vice versa.

Moreover, for any $\psi ,\varphi \in b[0,1]$, basic algebra and the triangle inequality for integrals tells us that

Working case by case, it is easy to check that for real numbers $a,b,c$ we always have

Combining (31.8) and (31.9) yields

Taking the supremum over $\pi$ now gives us

In other words, $Q$ is a contraction of modulus $\beta$ on the complete metric space $(b[0,1],\Vert \cdot \Vert )$.

Hence

• A unique solution $\overline{w}$ to the RWFE exists in $b[0,1]$.

• $Q^k\psi \to \overline{w}$ uniformly as $k\to \mathrm{\infty }$, for any $\psi \in b[0,1]$.

31.5.1. Exercise 1#

Use the default parameters and the .res_wage_operator() method to compute an optimal policy.

Your result should coincide closely with the figure for the optimal policy shown above.

Try experimenting with different parameters, and confirm that the change in the optimal policy coincides with your intuition.

31.6.1. Exercise 1#

This code solves the “Offer Distribution Unknown” model by iterating on a guess of the reservation wage function. You should find that the run time is much shorter than that of the value function approach in examples/odu_vfi_plots.jl.

sp = SearchProblem(pi_grid_size = 50)

phi_init = ones(sp.n_pi)
f_ex1(x) = res_wage_operator(sp, x)
w_bar = fixedpoint(f_ex1, phi_init).zero

plot(sp.pi_grid, w_bar, linewidth = 2, color = :black,
     fillrange = 0, fillalpha = 0.15, fillcolor = :blue)
plot!(sp.pi_grid, 2 * ones(length(w_bar)), linewidth = 0, fillrange = w_bar,
      fillalpha = 0.12, fillcolor = :green, legend = :none)
plot!(ylims = (0, 2),
      annotations = [(0.42, 1.2, "reject"),
          (0.7, 1.8, "accept")])

The next piece of code is not one of the exercises from QuantEcon – it’s just a fun simulation to see what the effect of a change in the underlying distribution on the unemployment rate is.

At a point in the simulation, the distribution becomes significantly worse. It takes a while for agents to learn this, and in the meantime they are too optimistic, and turn down too many jobs. As a result, the unemployment rate spikes.

The code takes a few minutes to run.

# Determinism and random objects.
using Random
Random.seed!(42)

# Set up model and compute the function w_bar
sp = SearchProblem(pi_grid_size = 50, F_a = 1, F_b = 1)
phi_init = ones(sp.n_pi)
g(x) = res_wage_operator(sp, x)
w_bar_vals = fixedpoint(g, phi_init).zero
w_bar = extrapolate(interpolate((sp.pi_grid,), w_bar_vals,
                                Gridded(Linear())), Flat())

# Holds the employment state and beliefs of an individual agent.
mutable struct Agent{TF <: AbstractFloat, TI <: Integer}
    _pi::TF
    employed::TI
end

Agent(_pi = 1e-3) = Agent(_pi, 1)

function update!(ag, H)
    if ag.employed == 0
        w = rand(H) * 2   # account for scale in julia
        if w >= w_bar(ag._pi)
            ag.employed = 1
        else
            ag._pi = 1.0 ./
                     (1 .+ ((1 - ag._pi) .* sp.g(w)) ./ (ag._pi * sp.f(w)))
        end
    end
    nothing
end

num_agents = 5000
separation_rate = 0.025  # Fraction of jobs that end in each period
separation_num = round(Int, num_agents * separation_rate)
agent_indices = collect(1:num_agents)
agents = [Agent() for i in 1:num_agents]
sim_length = 600
H = sp.G                 # Start with distribution G
change_date = 200        # Change to F after this many periods
unempl_rate = zeros(sim_length)

for i in 1:sim_length
    if i % 20 == 0
        println("date = $i")
    end

    if i == change_date
        H = sp.F
    end

    # Randomly select separation_num agents and set employment status to 0
    shuffle!(agent_indices)
    separation_list = agent_indices[1:separation_num]

    for agent in agents[separation_list]
        agent.employed = 0
    end

    # update agents
    for agent in agents
        update!(agent, H)
    end
    employed = Int[agent.employed for agent in agents]
    unempl_rate[i] = 1.0 - mean(employed)
end

plot(unempl_rate, linewidth = 2, label = "unemployment rate")
vline!([change_date], color = :red, label = "")

date = 20

date = 40
date = 60
date = 80
date = 100

date = 120
date = 140
date = 160
date = 180
date = 200

date = 220
date = 240
date = 260
date = 280

date = 300
date = 320
date = 340
date = 360

date = 380
date = 400
date = 420
date = 440

date = 460
date = 480
date = 500
date = 520

date = 540
date = 560
date = 580
date = 600