Data mining & data warehousing lecture notes

Data mining & data warehousing lecture notes in Data mining & Ware housing uploaded at P.R.Engineering college

Data mining & data warehousing lecture notes in Data mining & Ware housing at P.R.Engineering college uploaded by P.R.Engineering college Staff in lecturenotes.net

Summary / document preview text:

MC7403                               DATA WAREHOUSING AND DATA MINING                           L T P C 3 0 0 3
COURSE OBJECTIVES:
• To expose the students to the concepts of Data warehousing Architecture and Implementation  
• To Understand Data mining principles and techniques and Introduce DM as a cutting edge business intelligence  
• To learn to use association rule mining for handling large data
• To understand the concept of classification for the retrieval purposes
• To know the clustering techniques in details for better organization and retrieval of data
• To identify Business applications and Trends of Data mining
UNIT I                                       DATA WAREHOUSE                                                                         8
Data Warehousing - Operational Database Systems vs. Data Warehouses - Multidimensional Data Model - Schemas for Multidimensional Databases – OLAP Operations – Data Warehouse Architecture – Indexing – OLAP queries & Tools.
UNIT II                                      DATA MINING & DATA PREPROCESSING                                    9
Introduction to KDD process – Knowledge Discovery from Databases - Need for Data Preprocessing – Data Cleaning – Data Integration and Transformation – Data Reduction – Data Discretization and Concept Hierarchy Generation. 
UNIT III                                             ASSOCIATION RULE MINING                                                 8
     Introduction - Data Mining Functionalities - Association Rule Mining - Mining Frequent Item sets with    and  without Candidate Generation - Mining Various Kinds of Association Rules - Constraint-Based             Association Mining. 
UNIT IV                                          CLASSIFICATION & PREDICTION                                             10
classification vs. Prediction – Data preparation for Classification and Prediction – Classification by Decision Tree Introduction – Bayesian Classification – Rule Based Classification – Classification by Back Propagation – Support Vector Machines – Associative Classification – Lazy Learners – Other Classification Methods – Prediction – Accuracy and Error Measures – Evaluating the Accuracy of a Classifier or Predictor – Ensemble Methods – Model Section.
UNIT V                                                        CLUSTERING                                                                 10
Cluster Analysis: - Types of Data in Cluster Analysis – A Categorization of Major Clustering Methods – Partitioning Methods – Hierarchical methods – Density-Based Methods – Grid-Based Methods – Model-Based Clustering Methods – Clustering High- Dimensional Data – Constraint-Based Cluster Analysis – Outlier Analysis.
REFERENCES:
 1. Jiawei Han and Micheline  Kamber, “Data Mining Concepts and Techniques” Second Edition, Elsevier, Reprinted 2008. 
2. K.P. Soman,  Shyam Diwakar and V. Ajay,  “Insight into Data mining Theory and Practice”, Easter Economy Edition, Prentice Hall of India, 2006. 
3. G. K. Gupta,  “Introduction to Data Mining with Case Studies”, Easter Economy Edition, Prentice Hall of India, 2006. 
4. BERSON, ALEX & SMITH, STEPHEN J, Data Warehousing, Data Mining, and OLAP, TMH Pub. Co. Ltd, New Delhi, 2012
 5. Pang-Ning Tan, Michael Steinbach and Vipin Kumar, “Introduction to Data Mining”, Pearson Education, 2007 
6. PRABHU Data Warehousing, PHI Learning Private Limited, New Delhi, 2012, ,
 7. PONNIAH, PAULRAJ, Data Warehousing Fundamentals, John Wiley & Sons, New Delhi, 2011
 8. MARAKAS, GEORGE M, Modern Data Warehousing, Mining, and Visualiza Visualization, Pearson Education, 2011 MC7404 N

UNIT 1 - Data Warehouse

1.1  Data Warehouse:

A data warehouse is a subject-oriented, integrated, time-variant and non-volatile collection of

data in support of management's decision making process.

Subject-Oriented: A data warehouse can be used to analyze a particular subject area. For example, "sales" can be a particular subject.

Integrated: A data warehouse integrates data from multiple data sources. For example, source A and source B may have different ways of identifying a product, but in a data warehouse, there will be only a single way of identifying a product.

Time-Variant: Historical data is kept in a data warehouse. For example, one can retrieve data from 3 months, 6 months, 12 months, or even older data from a data warehouse. This contrasts with a transactions system, where often only the most recent data is kept. For example, a transaction system may hold the most recent address of a customer, where a data warehouse can hold all addresses associated with a customer.

Non-volatile: Once data is in the data warehouse, it will not change. So, historical data in a data warehouse should never be altered.

1.2  Data Warehouse Design Process:

A data warehouse can be built using a top-down approach, a bottom-up approach, or a combination of both.

The top-down approach starts with the overall design and planning. It is useful in cases where the technology is mature and well known, and where the business problems that must be solved are clear and well understood.

The bottom-up approach starts with experiments and prototypes. This is useful in the early stage of business modelling and technology development. It allows an organization to move forward at considerably less expense and to evaluate the benefits of the technology before making significant commitments.

In the combined approach, an organization can exploit the planned and strategic nature of the top-down approach while retaining the rapid implementation and opportunistic

application of the bottom-up approach.

The warehouse design process consists of the following steps:

Choose a business process to model, for example, orders, invoices, shipments, inventory, account administration, sales, or the general ledger. If the business process is organizational and involves multiple complex object collections, a data warehouse model should be followed. However, if the process is departmental and focuses on the analysis of one kind of business process, a data mart model should be chosen.

Choose the grain of the business process. The grain is the fundamental, atomic level of data to be represented in the fact table for this process, for example, individual transactions, individual daily snapshots, and so on.

Choose the dimensions that will apply to each fact table record. Typical dimensions are time, item, customer, supplier, warehouse, transaction type, and status.

Choose the measures that will populate each fact table record. Typical measures are numeric additive quantities like dollars sold and units sold.

1.3 Data Warehouse Architecture:

A Three Tier Data Warehouse Architecture:

Tier-1:

The bottom tier is a warehouse database server that is almost always a relational database system. Back-end tools and utilities are used to feed data into the bottom tier from operational databases or other external sources (such as customer profile information provided by external consultants). These tools and utilities perform data extraction, cleaning, and transformation (e.g., to merge similar data from different sources into a unified format), as well as load and refresh functions to update the data warehouse . The data are extracted using application program interfaces known as gateways. A gateway is
supported by the underlying DBMS and allows client programs to generate SQL code to be executed at a server.
Examples of gateways include ODBC (Open Database Connection) and OLEDB (Open Linking and Embedding for Databases) by Microsoft and JDBC (Java Database Connection).

This tier also contains a metadata repository, which stores information about the data warehouse and its contents.

Tier-2:

The middle tier is an OLAP server that is typically implemented using either a relational OLAP (ROLAP) model or a multidimensional OLAP.

OLAP model is an extended relational DBMS that maps operations on multidimensional  data to standard relational operations.

A multidimensional OL

Title:
Data mining & data warehousing lecture notes

Subject:
Data mining & Ware housing

Institute:
P.R.Engineering college

Data mining & data warehousing lecture notes

Data mining & data warehousing lecture notes in Data mining & Ware housing uploaded at P.R.Engineering college

Summary / document preview text:

Modal

Are you sure to delete this information ?