The notion of Big Data is identified with data of significant volume. They may come from social services, from many devices sensors, they may also be transaction data. The benefits that their analysis may bring are disproportionate to the used means, and such means are provided by Big Data solutions.
The notion of Big Data is identified with data of significant volume. They may come from social services, from many devices sensors, they may also be transaction data. They are characterized by variety (e.g. structuralized data coming from databases, or non-structuralized like audio files) and changeability (they are changed fast – the problem of fast processing). They may be tera- or even petabytes of data. The benefits that their analysis may bring are disproportionate to the used means, and such means are provided by Big Data solutions. The tools facilitate making accurate decisions and discovering new possibilities connected with a better processes optimization.
Big Data in Statistics Poland? A Recipe for Quickly Available Data
A necessary element of a well-functioning country is public statistics, and the main authority processing gathered data of this type is Statistics Poland.
The characteristic feature of data processed by public administration is to a great extent their bigger than average volume. In may be observed, for example, in public statistics or in the area of public registers. The existing technology, until Big Data tools appeared, had limited the use of data in analysis, applications and making right decisions. Unfortunately, data processing conducted by Statistics Poland cannot be described as Big Data because analysis published by the office are characterized by a substantial delay compared to a period they relate to. The use of Big Data tools would enable an improvement in time and accuracy of publications, which would enable making more accurate decisions and reacting faster to changes happening in the country.
Public Registers with Big Data Tools
The notion of public statistics is usually indirectly and directly connected with public registers which in Poland are led by different organizations of public administration, such as Statistics Poland, the Social Insurance Institution (ZUS) or Ministry of Health and their subordinate bodies. The registers usually store data on citizens, things and services. They are used for public tasks implementation, which they are obliged to do by appropriate legal regulations.
A vast majority of those data is stored in an electronic structuralized form, however, a small part is still stored only in a paper form. That is why their processing is much more difficult at present. The biggest electronic registers include the general Electronic Population Registration System (PESEL). The main data source for PESEL system are commune and voivodeship offices. Other public registers are The National Official Register of the Nationalised Industries Units (in short REGON) and National Court Register (KRS). The registers could be considered as meeting the Big Data criterium because of the data volume. However, the existing way of processing and relation bases structure excludes them from the notion. They are more of a transactional and batch processing character and are not used for more advanced analysis.
Big Data and Structural Data
Practical use of Big Data technology in public registers is desirable. It would enable not only analysis of separate registers, but mainly would join them together to enable a comprehensive analysis and making conclusions based on data from different sources at the same time. Big Data would also solve the problem of non-structural data processing. For example, there are medical registers, such as a register of medical care institutions, doctors or pharmacies registers, which have to communicate with registers such as PESEL or National Official Register of Territorial Division (in short TERYT). Too many divisions of such systems as well as the lack of consistency and communication between them lead to the lack of effective activity of the whole healthcare, and as a result to the like of quality of its services.
Big Data tools may influence improvements in the following areas:
• data monitoring in order to detect potential epidemiological risks early,
• fraud and irregularities detection in the tax system, supporting transport management (road planning, traffic control),
• improvement in the functioning of the system of education
• public safety by the analysis of citizens behaviour on the Internet as well as the analysis of recordings from the city surveillance system,
• better weather forecasting and early disaster prevention,
• analysis of the changing demand for employees and unemployment prevention in the labor market.
An intensive development of Big Data is mostly visible in commercial usage, where technological progress is implemented fast and it generates profits for investors. Public administration is an area where wide implementation of Big Data technology will bring measurable effects improving effectiveness and quality of its activity, and will enable costs reduction, so important for every tax-payer. Finally, it will lead to a better comfort of life of the society.