WOLFRAM

Wolfram Language & System Documentation Center

JSONLines (.jsonl)

Background & Context

    • Registered MIME types: application/jsonl
    • JSON Lines data format.
    • Consists of multiple JSON objects, one per line, representing individual data rows.
    • Commonly used for storing structured JSON data.
    • Text file format.

Import & Export

Import Elements

  • General Import elements:
  • "Elements" list of elements and options available in this file
    "Summary"summary of the file
    "Rules"list of rules for all available elements
  • Data representation elements:
  • "Data"two-dimensional array
    "Dataset"table data as a Dataset
    "EventSeries"table data as an EventSeries
    "Tabular"a Tabular object
    "TimeSeries"table data as a TimeSeries
  • Import by default uses the "Tabular" element.
  • Subelements for partial data import for the "Tabular" element can take row and column specifications in the form {"Tabular",rows,cols}, where rows and cols can be any of the following:
  • nnth row or column
    -ncounts from the end
    n;;mfrom n through m
    n;;m;;sfrom n through m with steps of s
    {n1,n2,…}specific rows or columns ni
  • Column specifications can also be any of the following:
  • "col"single column "col"
    {col1,col2,…}list of column names coli
  • Data descriptor elements:
  • "ColumnLabels"names of columns
    "ColumnTypes"association with data type for each column
    "Schema"TabularSchema object
  • Metadata elements:
  • "ColumnCount"number of columns stored in file
    "Dimensions"data dimensions
    "RowCount"number of rows stored in file
    "MetaInformation"metadata

Import Options

  • General Import options:
  • "BlockSize"1048576how many bytes to process at a time
    "Schema"Automaticschema used to construct Tabular object
    "TimeColumn"Automaticcolumn to use for times in "EventSeries" and "TimeSeries" elements
  • Possible settings for the "Schema" option include:
  • schemaa complete TabularSchema specification
    propvala schema property and value (see reference page for TabularSchema)
    <|"prop1"val1,…|>an association of schema properties and values

Export Options

  • Export options include:
  • "ExpressionFormattingFunction"Automatichow expressions stored in a Tabular object are converted to strings
  • "ExpressionFormattingFunction" can be set to the following values:
  • Automaticdefault conversion to string
    formany form supported by Format such as InputForm
    farbitrary function that converts an expression to a string

Examples

open all close all

Basic Examples  (3)

Import Tabular object from JSONLines file:

Wolfram Language code: Import["ExampleData/world_cups.jsonl"]

Import the file summary:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "Summary"]

Export Tabular object to JSONLines file:

Wolfram Language code: tabular = Import["ExampleData/world_cups.jsonl"];
Wolfram Language code: Export["file.jsonl", tabular]

Scope  (3)

Import  (3)

Show all elements available in the file:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "Elements"]

By default, a Tabular object is returned:

Wolfram Language code: Import["ExampleData/world_cups.jsonl"]//TabularQ

Import column types:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "ColumnTypes"]

Import Elements  (18)

"ColumnCount"  (1)

Get the number of columns:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "ColumnCount"]

"ColumnLabels"  (1)

Read column names:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "ColumnLabels"]

"ColumnTypes"  (1)

Import column types:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "ColumnTypes"]

"Data"  (3)

Get the data from a file:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "Data"]

Import only selected rows:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"Data", 1 ;; 2}]

Import only selected columns:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"Data", All, {1, 2}}]

Import selected columns using column names:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"Data", All, {"Year", "Host Country"}}]

"Dataset"  (3)

Get the data as a Dataset:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "Dataset"]

Import only selected rows:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"Dataset", 1 ;; 2}]

Import only selected columns:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"Dataset", All, {1, 2}}]

Import selected columns using column names:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"Dataset", All, {"Year", "Host Country"}}]

"Dimensions"  (1)

Import data dimensions:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "Dimensions"]

"EventSeries"  (1)

Import a JSONLines file as an EventSeries:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "EventSeries"]

Import a single row from a JSONLines file:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"EventSeries", 5}]

Import some specific rows from a JSONLines file:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"EventSeries", {1, 5, 7}}]

Import the first 10 rows of a JSONLines file:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"EventSeries", 1 ;; 10}]

Import only selected columns using column names:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"EventSeries", All, {"Year", "Host Country", "Winner"}}]

"RowCount"  (1)

Get the number of rows:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "RowCount"]

"Schema"  (1)

Get the TabularSchema object:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "Schema"]

"Summary"  (1)

Get the file summary:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "Summary"]

"Tabular"  (3)

Get the data from a file as a Tabular object:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "Tabular"]

Import only selected rows:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"Tabular", 1 ;; 2}]

Import only selected columns:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"Tabular", All, {1, 2}}]

Import selected columns using column names:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"Tabular", All, {"Year", "Host Country"}}]

"TimeSeries"  (1)

Import a JSONLines file as a TimeSeries:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", "TimeSeries"]

Import a single row from a JSONLines file:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"TimeSeries", 5}]

Import some specific rows from a JSONLines file:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"TimeSeries", {1, 5, 7}}]

Import the first 10 rows of a JSONLines file:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"TimeSeries", 1 ;; 10}]

Import only selected columns using column names:

Wolfram Language code: Import["ExampleData/world_cups.jsonl", {"TimeSeries", All, {"Year", "Host Country", "Winner"}}]

Import Options  (3)

"BlockSize"  (1)

Create a JSONLines file:

Wolfram Language code: Export["trees.jsonl", ResourceData["Sample Tabular Data: NYC Trees"]];

Import data using the default setting of "BlockSize"->2^20:

Wolfram Language code: RepeatedTiming[Import["trees.jsonl"];]

Import data faster by specifying the value of the "BlockSize" option:

Wolfram Language code: AssociationMap[First@RepeatedTiming[Import["trees.jsonl", "BlockSize" -> 2 ^ #];]&, {19, 21, 22}]

"Schema"  (1)

Wolfram Language code: file = Export["out.jsonl", Tabular[Association["RawSchema" -> Association["ColumnProperties" -> Association["A" -> Association["ElementType" -> "String"], "B" -> Association["ElementType" -> "String"]], "KeyColumns" -> None, "Backend" -> "WolframKernel"], "BackendData" -> Association["ColumnData" -> DataStructure["ColumnTable", {{TabularColumn[Association["Data" -> {{0, {0, 11, 22, 33, 44, 55}, "Jan 03 2006Jan 04 2006Jan 05 2006Jan 06 2006Jan 09 2006"}, {}, None}, "ElementType" -> "String"]], TabularColumn[Association[ "Data" -> {{3, {0, 5, 10, 15, 20, 25}, "11.8212.0412.0911.8812.43"}, {}, None}, "ElementType" -> "String"]]}}]]]]];

By default, column labels and their types stored in a file are used when Tabular or Dataset objects are imported:

Wolfram Language code: tabular = Import[file]; tabular["ColumnTypes"]

Use the "Schema" option to specify column labels and types:

Wolfram Language code: tabular = Import[file, "Schema" -> {"ColumnKeys" -> {"Date", "Value"}, "ElementType" -> {"Date" -> "Date", "Value" -> "Real32"}}]

"TimeColumn"  (1)

Export a Tabular object to a JSONLines file:

Wolfram Language code: file = Export["file.jsonl", Tabular[Association["RawSchema" -> Association["ColumnProperties" -> Association["Date" -> Association["ElementType" -> TypeSpecifier["Date"]["Integer32", "Day", "Gregorian", None]], "Value" -> Association["ElementType" -> "Real32"]], "KeyColumns" -> None, "Backend" -> "WolframKernel"], "Options" -> {}, "BackendData" -> Association["ColumnData" -> DataStructure["ColumnTable", {{TabularColumn[Association["Data" -> {5, {{NumericArray[{13150, 13151, 13152, 13153, 13156}, "Integer32"], {}, None}}, None}, "ElementType" -> "Date"["Integer32", "Day", "Gregorian", None]]], TabularColumn[Association[ "Data" -> {NumericArray[{11.819999694824219, 12.039999961853027, 12.09000015258789, 11.880000114440918, 12.430000305175781}, "Real32"], {}, None}, "ElementType" -> "Real32"]]}}]]]]];

By default, the time column is selected automatically for "TimeSeries" and "EventSeries" elements:

Wolfram Language code: Import[file, "TimeSeries"]

Use the "TimeColumn" option to specify the time column:

Wolfram Language code: Import[file, "TimeSeries", "TimeColumn" -> "Value"]

Export Options  (1)

"ExpressionFormattingFunction"  (1)

By default, Export uses different conversion to string depending on column types:

Wolfram Language code: tabular = Import["ExampleData/USstates.arrow", {"Tabular", 1 ;; 3}]
Wolfram Language code: ColumnTypes[tabular]
Wolfram Language code: ExportString[tabular, "JSONLines"]

Use "ExpressionFormattingFunction"->InputForm to get string versions of expressions suitable for input to the Wolfram Language:

Wolfram Language code: ExportString[tabular, "JSONLInes", "ExpressionFormattingFunction" -> InputForm]//Short[#, 5]&

Use an arbitrary function such as SpokenString:

Wolfram Language code: ExportString[tabular, "JSONLines", "ExpressionFormattingFunction" -> SpokenString]

Use a function that specifies different rules for different expression types:

Wolfram Language code: Clear[f] f[x_Image] := ImageIdentify[x]; f[x_] := ToString[x, OutputForm]
Wolfram Language code: ExportString[tabular, "JSONLines", "ExpressionFormattingFunction" -> f]

Applications  (1)

Import a large JSONLines file from Kaggle:

Wolfram Language code: data = DataConnectionObject[<|"ConnectionName" -> "Kaggle", "Location" -> "https://www.kaggle.com/datasets/devdope/900k-spotify?select=900k+Definitive+Spotify+Dataset.json"|>];
Wolfram Language code: Import[data, "JSONLines"]
Wolfram Language code: Dimensions[%]