This paper presents and evaluates a simple but very effective method to implement large data ware-houses on an arbitrary number of computers, achieving very high query execution performance and scalability. The data is distributed and processed in a potentially large number of autonomous computers using our technique called data warehouse striping (DWS). The major problem of DWS technique is that it would require a very expen-sive cluster of computers with fault tolerant capabilities to prevent a fault in a single computer to stop the whole system. In this paper, we propose a radically different approach to deal with the problem of the unavailability of one or more computers in the cluster, allowing the use of DWS with a very large number of inexpensive computers. The proposed approach is based on approximate query answering techniques that make it possible to deliver an approximate answer to the user even when one or more computers in the cluster are not available. The evaluation presented in the paper shows both analytically and experimentally that the approximate results obtained this way have a very small error that can be negligible in most of the cases.
Keywords
Data warehousing, distributed query optimization, data partitioning, performance optimization,
Subject
Data Warehousing
Journal
Journal of Intelligent Information Systems- Integrating Artificial Intelligence and Database Technologies, Vol. 19, #2, Elsevier Science Publication, September 2002
PDF File
Cited by
Year 2013 : 31 citations
1 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Handling big dimensions in distributed data warehouses using the DWS technique http://dl.acm.org/citation.cfm?id=1031770 M Costa, H Madeira Proceedings of the 7th ACM international … 2004 dl.acm.org
2 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 DWS-AQA: A cost effective approach for very large data warehouses http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1029676 J Bernardino, P Furtado… Database Engineering and … 2002 ieeexplore.ieee.org
3 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Providing ranked cooperative query answers using the metricized knowledge abstraction hierarchy http://www.sciencedirect.com/science/article/pii/S095741740500357X MK Shin, SY Huh, W Lee Expert Systems with Applications 2007 Elsevier
4 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Method using a hierarchy of servers for query processing of column chunks in a distributed column chunk data store http://www.google.com/patents?hl=en&lr;=&vid=USPAT7921131&id=JhlzAQAAEBAJ&oi=fnd&printsec=abstract RK Uppala US Patent 7,921,131 2011 Google Patents
5 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 A middle layer for distributed data warehouses using the DWS-AQA technique. http://solutions.criticalsoftware.com.br/upload_product/wp/A_middle_layer_for_distributed_data_warehouses_using_the_DWS-AQA_technique.pdf M Costa, J Vieira, J Bernardino, P Furtado… … 2003 solutions.criticalsoftware.com.br
6 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Algorithms for efficient processing of complex queries in node-partitioned data warehouses http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1319785 P Furtado … and Applications Symposium, 2004. IDEAS'04. … 2004 ieeexplore.ieee.org
7 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Efficient compression of text attributes of data warehouse dimensions http://link.springer.com/chapter/10.1007/11546849_35 J Vieira, J Bernardino, H Madeira Data Warehousing and Knowledge … 2005 Springer
8 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Scalable Query Infrastructure http://www.google.com/patents?hl=en&lr;=&vid=USPATAPP11537476&id=qyOoAAAAEBAJ&oi=fnd&printsec=abstract A Rowstron, R Mortier, A Donnelly… US Patent App. 11/ … 2006 Google Patents
9 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 System of a hierarchy of servers for query processing of column chunks in a distributed column chunk data store http://www.google.com/patents?hl=en&lr;=&vid=USPAT7860865&id=w0bwAAAAEBAJ&oi=fnd&printsec=abstract RK Uppala US Patent 7,860,865 2010 Google Patents
10 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Accuracy estimation in approximate query processing http://www.wseas.us/e-library/conferences/2010/Corfu/COMPUTERS/COMPUTERS2-08.pdf C Dell'Aquila, F Di Tria, E Lefons… Proceedings of the 14th … 2010 wseas.us
11 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Data warehouse access using multi-agent system http://link.springer.com/article/10.1007/s10619-009-7036-9 N Kolsi, A Abdellatif, K Ghedira Distributed and Parallel Databases 2009 Springer
13 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Building OLAP tools over large databases http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.109.4447&rep=rep1&type=pdf R Oliveira, J Bernardino Proceedings of IADIS Virtual Multi Conference 2006 Citeseer
14 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Metrics for approximate query engine evaluation http://dl.acm.org/citation.cfm?id=2245448 F Di Tria, E Lefons, F Tangorra Proceedings of the 27th Annual ACM … 2012 dl.acm.org
15 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Probabilistic model for accuracy estimation in approximate monodimensional analyses http://www.wseas.us/e-library/transactions/computers/2010/88-281.pdf C Dell'Aquila, F Di Tria, E Lefons… WSEAS Transactions on … 2010 wseas.us
16 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Multi agent systems in distributed datawarehousing http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5640491 TP Latchoumi, R Sunitha Computer and Communication … 2010 ieeexplore.ieee.org
17 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Spatial Telemetric Data Warehouse and Software Agents as Environment to Distributed Execute SQL Queries http://www.proceedings2006.imcsit.org/pliks/122.pdf M Gorawski, E P?uciennik Proceedings of the … 2006 proceedings2006.imcsit.org
18 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Exploiting data access for dynamic fragmentation in data warehouse http://inderscience.metapress.com/index/N2583608V2G35686.pdf H Derrar, M Ahmed?Nacer, O Boussaid International Journal of … 2013 Inderscience
19 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 An Analysis of Data Warehouse Research http://www.academic-journals.org/EC/v4n1-2006-spring/ECSv4n1p1.pdf HY Lin, PY Hsu, YM Su Electronic Commerce Studies 2006 academic-journals.org
20 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 System and method for adding a storage server in a distributed column chunk data store http://www.google.com/patents?hl=en&lr;=&vid=USPAT8214388&id=hHMfAgAAEBAJ&oi=fnd&printsec=abstract RK Uppala US Patent 8,214,388 2012 Google Patents
21 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Method for query processing of column chunks in a distributed column chunk data store http://www.google.com/patents?hl=en&lr;=&vid=USPAT7921087&id=5BRzAQAAEBAJ&oi=fnd&printsec=abstract RK Uppala US Patent 7,921,087 2011 Google Patents
22 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 System for query processing of column chunks in a distributed column chunk data store http://www.google.com/patents?hl=en&lr;=&vid=USPAT7921132&id=JxlzAQAAEBAJ&oi=fnd&printsec=abstract RK Uppala US Patent 7,921,132 2011 Google Patents
23 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 AGENT BASED DATA STORAGE AND DISTRIBUTION IN DATA WAREHOUSES http://www.worldscientific.com/doi/abs/10.1142/S0218194008003842 N Kolsi, A Abdellatif, K Ghedira International Journal of Software … 2008 World Scientific
24 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Data Warehouse System Evaluation and Selection Decisions http://thesis.lib.ncu.edu.tw/ETD-db/ETD-search/view_etd?URN=91441002 HY Lin Publication/NA 2006 thesis.lib.ncu.edu.tw
25 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 The F&A Methodology and Its Experimental Validation on a Real-Life Parallel Processing Database System http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6245571 L Bellatreche, S Benkrid, A Crolotte… … (CISIS), 2012 Sixth … 2012 ieeexplore.ieee.org
26 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Cost-Effective Data Allocation in Data Warehouse Striping http://www.doaj.org/doaj?func=fulltext&aId=1177338 R Almeida, J Vieira, M Vieira, H Madeira… International … 2012 doaj.org
27 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 A Query Cache Tool for Optimizing Repeatable and Parallel OLAP Queries http://link.springer.com/chapter/10.1007/978-3-642-03573-9_11 RJ Santos, J Bernardino Database and Expert Systems Applications 2009 Springer
28 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Particle swarm optimisation for data warehouse logical design http://inderscience.metapress.com/index/74N7768848255414.pdf HÁ Derrar, M Ahmed?Nacer, O Boussaid International Journal of Bio- … 2012 Inderscience
29 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 Leveraging 24/7 Availability and Performance for Distributed Real-Time Data Warehouses http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6340224 RJ Santos, J Bernardino… Computer Software and … 2012 ieeexplore.ieee.org
30 "Approximate Query Answering Using Data Warehouse Striping
" 4543013994316329237 [CITATION][C] Uma Solução para Business Intelligence http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6340224 Authors/NA Publication/NA Year/NA Publisher/NA
Year 2009 : 1 citations
Nader Kolsi, Abdelaziz Abdellatif, and Khaled Ghedira, "Data warehouse access using multi-agent system?, Journal of Distributed and Parallel Databases, Volume 25, Numbers 1-2 / April, 2009
Year 2008 : 1 citations
Kolsi, Nader, Abdellatif, Abdelaziz, Ghedira, Khaled, "Agent Based Data Storage and Distribution in Data Warehouses?, International Journal of Software Engineering and Knowledge Engineering. Vol. 18, no. 5, pp. 597-617. Aug. 2008
Year 2006 : 2 citations
Marcin Gorawski, Ewa Płuciennik, "Spatial Telemetric Data Warehouse and Software Agents as Environment to Distributed Execute SQL Queries?, proceedings of the International Multiconference on Computer Science and Information Technology, pp. 243 " 252, 2006
Hua-Yang Lin, "Data Warehouse System Evaluation and Selection Decisions?, PhD thesis, 2006 (http://thesis.lib.ncu.edu.tw/ETD-db/ETD-search/getfile?URN=91441002&filename=91441002.pdf).
