我想把下面的字符串变量转换成spark上的dataframe.
val jsonStr = "{ "metadata": { "key": 84896, "value": 54 }}"
我知道如何从json文件创建数据帧.
sqlContext.read.json("file.json")
but I don't know how to create dataframe from string variable.
如何将json字符串变量转换为dataframe.
我想把下面的字符串变量转换成spark上的dataframe.
val jsonStr = "{ "metadata": { "key": 84896, "value": 54 }}"
我知道如何从json文件创建数据帧.
sqlContext.read.json("file.json")
but I don't know how to create dataframe from string variable.
如何将json字符串变量转换为dataframe.
For Spark 2.2+:
import spark.implicits._
val jsonStr = """{ "metadata": { "key": 84896, "value": 54 }}"""
val df = spark.read.json(Seq(jsonStr).toDS)
Spark 2.1.x美元:
val events = sc.parallelize("""{"action":"create","timestamp":"2016-01-07T00:01:17Z"}""" :: Nil)
val df = sqlContext.read.json(events)
提示:这是使用
sqlContext.read.json(jsonRDD: RDD[Stirng])重载.
val jsonStr = """{ "metadata": { "key": 84896, "value": 54 }}"""
val rdd = sc.parallelize(Seq(jsonStr))
val df = sqlContext.read.json(rdd)