Statistics Canada workflow example
19 March 2025 - Written by ML
In this report we present a workflow for pulling Canadian statistics and demonstrate how to extract data on employment in the oil and gas extraction industry. The approach uses Statistics Canada’s bulk delta daily data file1 along with data from the Statistics Canada website. This combined approach helped to understand the data file structure and rapidly identify the specific vector code2 for our series of interest
Statistics Canada produces a daily file called delta that contains statistics data. When downloaded as a ZIP file, the delta3 package includes the data, metadata, and a code file. Content within the code file can be used to map data onto both meta and data file fields. And the meta file is connected to the data file through a common field - productId which is the numeric table identifier.
After unzipping the container, the metadata and code files (which are in XML format) are converted into CSV format.
The Statistics Canada website is then used to identify the series of interest. And the vector ID is found from within the created database file that contains the specific series data.
The data file includes several key fields such as:
| Metric | Value |
|---|---|
| Number of Rows | 61847371 |
| Number of Columns | 13 |
| Memory Usage | 3.9 Gb |
| Column | Data_Type | Sample_Value | |
|---|---|---|---|
| productId | productId | integer | 10100139 |
| coordinate | coordinate | character | 1.12.0.0.0.0.0.0.0.0 |
| vectorId | vectorId | integer | 39050 |
| refPer | refPer | character | 2025-03-12 |
| refPer2 | refPer2 | logical | NA |
| symbolCode | symbolCode | integer | 0 |
| statusCode | statusCode | integer | 0 |
| securityLevelCode | securityLevelCode | integer | 0 |
| value | value | numeric | 2.9991 |
| releaseTime | releaseTime | character | 2025-03-17T08:30 |
| scalarFactorCode | scalarFactorCode | integer | 0 |
| decimals | decimals | integer | 4 |
| frequencyCode | frequencyCode | integer | 1 |
The metadata file provides supplementary information about all series
related to the table of interest. It includes details on the dimensions
(e.g., geography, industry classification, release date) that can be
used to filter and link data. The key linking field is
productId, which connects the data file with the meta
file.
The key meta file field of interest is DimensionsSummary that provides an understanding to the coordinate field in the data file. In the case of the employment table there are four dimensions listed including: Geography (Pos 1): 1 members; Estimate (Pos 2): 2 members; North American Industry Classification System (NAICS) (Pos 3): 435 members; Release (Pos 4): 120 members.
By using the website for this particular series these categories are clearly shown. For our example we have selected:
Once the criteria were selected in the website the data could be downloaded. Of the various download file options the one that contained the data in database format contained the vector code which is the unique identifier for our series of interest.
After obtaining the vector code we could then extract the precise series with its reference date and value and place that into an R dataframe for further analysis and visualization.
| Metric | Value |
|---|---|
| Number of Rows | 1 |
| Number of Columns | 21 |
| Memory Usage | 29.7 Kb |
| Column | Data_Type | |
|---|---|---|
| ProductID | ProductID | integer |
| CansimId | CansimId | character |
| CubeTitleEn | CubeTitleEn | character |
| CubeTitleFr | CubeTitleFr | character |
| CorrectionID | CorrectionID | logical |
| CorrectionDate | CorrectionDate | logical |
| CorrectionNoteEn | CorrectionNoteEn | logical |
| CorrectionNoteFr | CorrectionNoteFr | logical |
| CubeStartDate | CubeStartDate | IDate, Date |
| CubeEndDate | CubeEndDate | IDate, Date |
| NbSeriesCube | NbSeriesCube | integer |
| NbDatapointsCube | NbDatapointsCube | integer |
| ArchiveStatusCode | ArchiveStatusCode | integer |
| SubjectCode | SubjectCode | logical |
| SurveyCode | SurveyCode | logical |
| frequencyEn | frequencyEn | character |
| FrequencyFr | FrequencyFr | logical |
| responseStatusCode | responseStatusCode | integer |
| DimensionsSummary | DimensionsSummary | character |
| FootnotesEn | FootnotesEn | character |
| FootnotesFr | FootnotesFr | character |
The next step in this approach is to find an efficient workflow for rapid identification of all unique vectors for each table. And then to establish a work flow that would allow iteration over all dimension series for certain statistics. For example if we wanted to understand variance in employment for all employment sectors across each month per year.
[as of 2025-03-17]↩︎
[1645360662]↩︎
[url = “https://www150.statcan.gc.ca/n1/delta/20250312.zip”]↩︎