How to listen to noise and analyse data: the ‘noisy’ data analysis

Australia is a country of noisy data, says a new report from the Financial Review, and the Government’s efforts to cut emissions and improve the quality of the data they collect have been undermined by the use of data that is “noisy”.

The report by Australian data scientist Paul McGlinchey, a former chief data officer at the Federal Government, shows the data produced by Australia’s public and private sector is “troublingly inconsistent” and that “there is a need to change the way data is produced and analysed”.

The findings are based on the review’s annual “noise survey” that takes into account data quality and data quality control, but also the volume and quality of data generated by the private sector.

A total of 22 questions were asked in the survey, which was undertaken in February and March this year.

The report says that the response rate was 70 per cent, which is well above the national average of 50 per cent.

It also says that there is a significant lack of data quality in the data that Australia produces and that data quality is being undermined by “low level data quality”, which it calls “noising” in data.

It adds that Australia’s data collection system is “unresponsive to data quality concerns”, and “data quality is often ignored or poorly recorded”.

It recommends that data “analysts and data analysts work together to produce data that has more meaningful information and is easier to understand”.

“We have a responsibility to ensure that the quality and quality control of data produced and consumed by the Australian public and businesses is the highest possible,” Ms McGlinchy says.

“There is a real need for better data quality controls and better data reporting.

Data should be available to the public to inform policy and decision making.”

The report also highlights that Australia is “failing to meet its obligation to protect public health”, and recommends that the Federal Data Protection Commissioner look into the way that data is collected and used.

The review is based on a survey of about 5,000 Australians, and was carried out by the data scientist for the Commonwealth of Australia, the Office of National Statistics and the Australian Bureau of Statistics.

“This is a serious report on how Australia is failing to meet our national health and environment responsibilities,” Mr McGlincon says.

The results of the survey show that: Australians are less satisfied with their data quality than they were a year ago;