我正在try 建立一个Sagemaker管道,有2个步骤:预处理,然后训练一个射频模型. 第一步产生3个输出:scaled_data.csv、train.csv和est.csv.第二步是训练和测试CSV来训练射频模型.在运行步骤2时出现错误,指出"TypeError:类型属性的对象不是JSON Serializable".
以下是我设置管道步骤的代码:
# upload data from local path to default bucket with prefix raw_data
WORK_DIRECTORY = "data"
input_data = sagemaker_session.upload_data(
path="{}/{}".format(WORK_DIRECTORY, "dataset.csv"),
bucket=bucket,
key_prefix="{}/{}".format(prefix, "input_data"),
)
- 设置第一步(扩展步骤)
scaling_processor = SKLearnProcessor(
framework_version=FRAMEWORK_VERSION,
instance_type="ml.m5.4xlarge",
instance_count=processing_instance_count,
base_job_name="data-process",
role=role,
sagemaker_session=pipeline_session,
)
scaling_processor_args = scaling_processor.run(
inputs=[
ProcessingInput(source=input_data, destination="/opt/ml/processing/input"),
],
outputs=[
ProcessingOutput(output_name="scaled_data", source="/opt/ml/processing/output/scaled_data/"),
ProcessingOutput(output_name="train", source="/opt/ml/processing/output/train/"),
ProcessingOutput(output_name="test", source="/opt/ml/processing/output/test/") ],
code="scripts/preprocess.py",
)
step_process = ProcessingStep(name="DataProcess", step_args=scaling_processor_args)
- 设置第二步(RF训练-BYO模式),这里是出现错误的地方:
estimator_cls = sagemaker.sklearn.SKLearn
FRAMEWORK_VERSION = "0.23-1"
rf_processor = FrameworkProcessor(
estimator_cls,
FRAMEWORK_VERSION,
role = role,
instance_count=1,
instance_type='ml.m5.2xlarge',
base_job_name='rf-modelling'
)
rf_processor_args = rf_processor.run(
inputs=[
ProcessingInput(source=step_process.properties.ProcessingOutputConfig.Outputs["train"].S3Output.S3Uri,
destination="/opt/ml/processing/input"),
ProcessingInput(source=step_process.properties.ProcessingOutputConfig.Outputs["test"].S3Output.S3Uri,
destination="/opt/ml/processing/input"), ],
outputs=[
ProcessingOutput(output_name="rf_model",source = "/opt/ml/processing/output/")
],
code="scripts/train.py",
)
step_train = ProcessingStep(name="RFTrain", step_args=rf_processor_args)
在运行步骤2时出现错误,指出"TypeError:类型属性的对象不是JSON Serializable".问题出在我为RF_PROCESSOR_ARGS中的第二步设置处理输入的那一行. 你知道是什么原因导致这个错误吗?