Typescript 在不使用任何对象的情况下访问对象的子对象

I'm in the process of applying better TypeScript to a project. There is a complex object coming from my backend which is a class with dozens of attributes, most of which are classes of their own. We have a form with a common change handler that currently has a few anys in it to make it work, but I'd like to fix that with generics.

This is a simplified version of the pattern in my application, but I am able to reproduce the same errors I'm seeing:

class PhoneNumber {
  prefix: number;
  exchange: number;
  line: number;
}

class Address {
  address1: string;
  address2: string;
  city: string;
}

class User {
  phone: PhoneNumber;
  address: Address;
}

const user = new User();

function setValue<A extends keyof User, T extends User[A], K extends keyof T, V extends T[K]>(
  value: V,
  fieldType: new () => T,
  fieldName: A,
  fieldAttribute: K
) {

  if (!user[fieldName]) {
    user[fieldName] = new fieldType();
  }

  const field = user[fieldName];

  // error here.
  field[fieldAttribute] = value;
}

setValue(408, PhoneNumber, 'phone', 'prefix');

I tried a few things, and this is as close as I'm able to get to a solution. I'm able to inspect the function call at the end in my IDE, and it looks like the generics are being filled with something sane: function setValue<"phone", PhoneNumber, "prefix", number>(), but then I get an error during compilation:

TS2536: Type 'K' cannot be used to index type 'User[A]'.

Am I going about this the wrong way? If I can't figure this out, I'm going to end up just splitting out the different types into separate handlers.

There's also a case where some of the fields are arrays of objects, which adds another wrinkle to the whole thing.