1. Startups

Between Official Statistics and Big Data Technology

Big data technology is not a substitute for official statistical data, but complements it. There are several traditional approaches that big data can replace

The popularity of big data technology has been increasing along with the volume of data that continues to increase because online-based services also continue to emerge. The ease with which it offers to collect huge amounts of data, even though it is not structured, and can be processed to assist business decision making has also played a role in increasing its popularity. However, what about the traditional data collection methods adopted by official statistical bodies such as BPS. Can big data technology replace it?

This is the question that the Associate Professor (Head Lector) of STIS BPS is trying to answer Faithful Pramana at the Big Data Indonesia Conference held by idBigData for two days (7-8 December 2016) at the BPPT Auditorium, Jakarta.

“Maybe not, we replace official statistics with big data? Data [big data] that's already huge, maybe not to get the information needed for the government to make policies? There is no need for official statistics anymore, there is no need for a census, because it is expensive and uses the state budget," said Setia.

Before discussing further, Setia provides an explanation of the difference between official statistics and big data in the data collection process as shown in the following table:

Official StatisticsBig Data
Structural and Planned ProductsLargely unstructured / unfiltered data “Data exhaust”, ie, by product of digital products (transactions, web, social media, sensors)
Methodological and clear conceptPoor analytics
regulatedUnregulated
Macro-level, but typically based on high volume primary dataMicro-level, huge volume with high velocity (or frequency) and variety
high costGenerally little or no cost
Centralized; point in timeDistributed; real-time

Setia said, “Looking at the two comparisons, there are several possible approaches that the official statistics cannot be replaced [by big data] because there are standards, but information from official statistics can be paired or paired with information from big data. So, big data can be a complement to official statistics.”

In terms of its complementary role, Setia explained further, big data can provide variables to help BPS classify better for sample surveys. It can also help improve survey forecasts, helping to balance the data non-response such as company data, helps to check BPS forecasts, helps speed up analysis results so as to speed up data release, and helps to scale up and provide smaller data estimates.

Comparison of BPS and Twitter data for the movement patterns of commuter line users / DailySocial

In an effort to prove this concept, BPS itself has compared their survey data with data obtained from online services. For example, data on the movement of commuter line users surveyed by BPS with data obtained from Twitter. As a result, it looks like there are similarities in patterns and trends.

“There are indeed similarities in patterns and trends that can be seen. […] However, big data is not a substitute, but it is more complementary [from official statistics] and check also complement if there are deficiencies. […] There are some that can indeed be – replace, but that remains to be discussed. [For now] There are still many things that need to be explored which ones can use big data, which ones can't. Then, the information must have contributions from the stakeholders stakeholders related," he said.

- Disclosure: DailySocial is a media partner of the 2016 Big Data Indonesia Conference

Are you sure to continue this transaction?
Yes
No
processing your transactions....
Transaction Failed
try Again

Sign up for our
newsletter

Subscribe Newsletter
Are you sure to continue this transaction?
Yes
No
processing your transactions....
Transaction Failed
try Again