By WalkingTree November 10, 2020
Data profiling is a process of examining, analyzing, and reviewing data to collect statistics surrounding the quality of a dataset. Data quality refers to the accuracy, consistency, validity, and completeness of data. The main benefit of performing data profiling is to improve the quality of your data.
Types of data profiling techniques –
- Completeness — how many nulls are in a column?
- Uniqueness — how many unique values are in a column? Any duplicates? Is it allowed to have duplicates?
- Value distribution — The distribution of records across different values for a specific attribute
- Range — Finding the minimum, maximum, and average value within the column
Data Profiling and Power BI
Power BI gives a possibility, not only to create appealing visuals but also to shape your data prior to that. When it comes to data profiling, Power BI has a lot to offer. It enables data profiling in a user-friendly way since it has some built-in functionalities that will make Data profiling much easier.
Let’s take a look at some of the tips regarding Data profiling:
- The first thing to do is to open the data model in a Model tab within the Power BI Desktop. This way, developers can be familiar with the tables and identify key columns and relationships between the tables. That gives an overview of the data model.
- The next step is turning on the Power Query editor, where most of the data profiling occurs. Under the View tab, turn on Column distribution, Column profile, and Column quality options. Once you select a specific column, you should be able to see the data profiled. Power BI helps spot the outliers or any other unexpected behavior of data.
Read on to know more about Data profiling and how Power BI can be helpful.
At WalkingTree, we have been rapidly transforming our development, testing, building and deployment processes using some of the…
Microservices is the latest norm for enterprise development and many newly built applications are inherently adopting its core…
In my previous blog, we discussed the ‘Log management of Microservices using ELK’ in data center kind…
As we watch recent architecture trends in the enterprise app development area, we observe that Microservices…
In my previous blog, we discussed the importance of inter-service communication and especially asynchronous communication in Microservices….