Now that you have performed some data transformation exercises, it is a good time to read about some applicable transformation and data management concepts. Transformation As you progressed through the exercises that transformed the...
Read More
Normalize and Denormalize Values– Transform, Manage, and Prepare Data
Notice that the input for normalization is the same as the data shown in the previous data table. The aspect of the resulting normalized data you may notice first is the existence of the...
Read More
Azure Cosmos DB—Shred JSON– Transform, Manage, and Prepare Data
FIGURE 5.23 Shredding JSON with Azure Cosmos DB The query you executed in step 4 begins with a SELECT, which is followed by the OPENROWSET that contains information about the PROVIDER, CONNECTION, and OBJECT.SELECT...
Read More
Split Data – Transform, Manage, and Prepare Data
FIGURE 5.21 Splitting the data source—Projection tab FIGURE 5.22 Splitting the data sink—Optimize tab In Exercise 5.6 you created a data flow that contains a source to import a large CSV file from ADLS....
Read More
Transform Data Using Stream Analytics – Transform, Manage, and Prepare Data
Remember, as you begin this section, that Chapter 7, “Design and Implement a Data Stream Processing Solution,” is devoted to data streaming. The content in this section will therefore target the Azure Stream Analytics...
Read More
Encode and Decode Data– Transform, Manage, and Prepare Data
The output is SQL_Latin1_General_CP1_CI_AS, which is the default (refer to Figure 3.28). GO INSERT INTO [dbo].[ENCODE] ([ENCODE_ID], [ENCODE]) VALUES (1, ‘殽’)INSERT INTO [dbo].[ENCODE] ([ENCODE_ID], [ENCODE]) VALUES (2, ‘Ž’)INSERT INTO [dbo].[ENCODE] ([ENCODE_ID], [ENCODE]) VALUES (3,...
Read More
Transform Data Using Apache Spark—Azure Synapse Analytics – Transform, Manage, and Prepare Data-1
Transform Data Using Apache SparkApache Spark can be used in a few products running on Azure: Azure Synapse Analytics Spark pools, Azure Databrick Spark clusters, Azure HDInsight Spark clusters, and Azure Data Factory. The...
Read More
Transform Data Using Azure Synapse Pipelines – Transform, Manage, and Prepare Data-3
One action you may have noticed in Exercise 5.1 is that you used the existing pipeline that you created in Exercise 4.13. That pipeline performed one activity, which was to copy data from the...
Read More
Transform Data Using Azure Synapse Pipelines – Transform, Manage, and Prepare Data
It does provide some benefit to understand the structure of the data you must ingest, transform, and progress through the other Big Data pipeline stages. It is helpful to know because as you make...
Read More
Transform Data Using Apache Spark—Azure Databricks – Transform, Manage, and Prepare Data
The Azure Databricks workspace should resemble Figure 5.12. FIGURE 5.12 Transforming data using an Apache Spark Azure Databricks workspace The first important point for Exercise 5.4 has to do with the location of the...
Read More