Structured and Unstructured Data

Structured and Unstructured Data

Data can be stored and presented in many different ways. As a data analyst,
one of the first things you need to understand is how information is organized.
The organization of data affects how easily it can be stored, searched, cleaned,
processed, analyzed and visualized.

Two major categories are structured data and
unstructured data. Between them is another important category
called semi-structured data.

Understanding these three categories is essential because modern organizations
work with information from databases, spreadsheets, websites, applications,
documents, social media, images, audio and video.

1. What Is Structured Data?

Structured data is data that is organized according to a clearly defined
structure or schema. It normally follows a predictable format and can easily
be arranged into rows and columns.

A traditional database table is one of the most common examples of structured
data.

For example:

Each column has a defined meaning and each row represents a record.
This makes the information relatively easy to search and analyze.

2. Characteristics of Structured Data

Structured data usually has the following characteristics:

• It follows a predefined structure.
• It is usually organized into rows and columns.
• Columns normally have defined names.
• Data types can be defined for each field.
• It can be easily queried using database systems.
• It is relatively easy to validate and clean.
• It works well with traditional analytical tools.

3. Examples of Structured Data

Common examples include:

• Customer databases
• Bank transaction records
• Student records
• Employee databases
• Inventory tables
• Sales records
• Payroll records
• Excel spreadsheets
• SQL database tables

4. Structured Data in a Database

Consider an online store with a customer table:

customer_id | name     | email              | age
---------------------------------------------------
1           | Ade      | ade@example.com    | 28
2           | Grace    | grace@example.com  | 31
3           | John     | john@example.com   | 25

The database knows what each field represents.
For example:

• customer_id can be an integer.
• name can be text.
• email can be text.
• age can be an integer.

This predictable structure allows SQL queries to retrieve specific information.

Example SQL Query

SELECT name, age
FROM customers
WHERE age > 25;

The database can efficiently find customers whose age is greater than 25
because the data follows a known structure.

5. What Is a Schema?

A schema describes how data is organized.
It defines elements such as:

• Table names
• Column names
• Data types
• Relationships
• Constraints

For example, a student database may define:

students

id       INTEGER
name     VARCHAR
age      INTEGER
email    VARCHAR
score    DECIMAL

The schema provides a predictable framework for storing information.

6. Primary Keys

A primary key is a field used to uniquely identify a record in a table.

For example:

student_id
1001
1002
1003

Each student can have a unique student ID. This makes it possible to identify
one particular record without confusion.

7. Foreign Keys

A foreign key is a field that connects a record in one table to a record
in another table.

For example, a training system may have:

students
---------
student_id
name

enrollments
-----------
enrollment_id
student_id
course_id

The student_id in the enrollments table can reference the
student record in the students table.

This allows relational databases to connect related information.

8. Advantages of Structured Data

• Easy to organize.
• Easy to search.
• Easy to query.
• Easy to validate.
• Works well with SQL.
• Suitable for statistical analysis.
• Easy to visualize.
• Can be efficiently stored in relational databases.

9. Limitations of Structured Data

Structured data is powerful, but it also has limitations.

• It requires a defined structure.
• Changing the structure can sometimes require database modifications.
• It may not naturally represent images, videos or complex documents.
• Some real-world information does not fit neatly into rows and columns.

10. What Is Semi-Structured Data?

Semi-structured data does not follow the strict tabular structure of a
traditional relational database, but it still contains some organizational
features such as tags, keys, attributes or metadata.

Common examples include:

• JSON
• XML
• HTML
• Email metadata
• Log files

Semi-structured data is more flexible than traditional structured data.

11. JSON Data

JSON stands for JavaScript Object Notation. It is widely
used for exchanging information between web applications and systems.

Example:

{
    "name": "Ade",
    "age": 28,
    "course": "Data Analysis",
    "skills": [
        "Excel",
        "SQL",
        "Python"
    ]
}

Although this is not a traditional table, the keys and values provide
an identifiable structure.

12. XML Data

XML stands for Extensible Markup Language.
It uses tags to organize information.

<student>
    <name>Ade</name>
    <age>28</age>
    <course>Data Analysis</course>
</student>

The tags provide structure while still allowing the information to be
nested in flexible ways.

13. HTML as Semi-Structured Data

documents contain tags that organize information on web pages.
For example:

<article>
    <h1>Data Analysis</h1>
    <p>Data analysis involves examining data to discover useful insights.</p>
</article>

The document is not a simple database table, but its tags provide meaningful
structure.

14. Email Data

An email may contain both structured and unstructured elements.

For example, email metadata may contain:

• Sender
• Recipient
• Date
• Subject

These fields have recognizable structure. However, the actual email message
may contain free-form text.

This makes email a good example of information that can contain multiple
forms of data.

15. Advantages of Semi-Structured Data

• More flexible than rigid database tables.
• Supports nested information.
• Works well with modern web applications.
• Can represent complex relationships.
• Easy to exchange between different systems.

16. Limitations of Semi-Structured Data

• It may require parsing before analysis.
• Different records may contain different fields.
• Data validation can be more complicated.
• Large nested structures can be difficult to analyze directly.

17. What Is Unstructured Data?

Unstructured data is information that does not follow a predefined tabular
schema or consistent data model.

Examples include:

• Images
• Videos
• Audio recordings
• PDF documents
• Word documents
• Customer reviews
• Social media posts
• Emails
• Scanned documents

Unstructured data can contain valuable information, but extracting that
information often requires specialized techniques.

18. Examples of Unstructured Data

Images

Photographs, scanned documents, product images and medical images are examples
of unstructured data.

Audio

Telephone conversations, interviews, podcasts and voice recordings contain
audio information.

Video

Security camera recordings, training videos, advertisements and social media
videos are examples of video data.

Documents

Reports, contracts, letters and PDF files may contain large amounts of
information that is not organized into a database table.

Social Media

Posts, comments, photographs, videos and user-generated content can contain
different forms of unstructured information.

19. Why Unstructured Data Is Difficult to Analyze

Unstructured data is challenging because computers cannot always immediately
understand its meaning.

For example, consider the sentence:

The customer said that the new laptop is excellent but the battery does not
last long enough.

A human can easily identify the customer’s positive and negative opinions.
A computer may require Natural Language Processing techniques to identify
these sentiments automatically.

20. Metadata

Metadata is information that describes other data.

For example, an image may contain metadata such as:

• File name
• File size
• Creation date
• Camera model
• Image dimensions
• Location information, when available

Metadata can make unstructured information easier to organize and search.

21. Structured vs Semi-Structured vs Unstructured Data

22. Data Warehouses

A data warehouse is a centralized system designed to store
and analyze large amounts of structured and processed data.

Organizations commonly use data warehouses for:

• Business intelligence
• Reporting
• Historical analysis
• Dashboard development
• Decision making

Data warehouses generally work best with organized analytical data.

23. Data Lakes

A data lake is designed to store large amounts of data in
different formats, including structured, semi-structured and unstructured data.

A data lake may contain:

• Database exports
• CSV files
• JSON files
• Images
• Audio
• Video
• Application logs

The flexibility of data lakes makes them useful when organizations want to
store information before deciding exactly how it will be analyzed.

24. ETL and ELT

Data analysts and data engineers often work with processes that move data
from its original sources into systems where it can be analyzed.

ETL means:

• Extract
• Transform
• Load

Data is extracted from a source, transformed into a useful format and then
loaded into a target system.

ELT means:

• Extract
• Load
• Transform

In ELT, the data is first loaded into the target platform and transformation
may happen later.

25. Converting Unstructured Data into Structured Data

Data analysts and data professionals often need to transform unstructured
information into a format that can be analyzed.

For example, an organization may receive hundreds of customer comments.
The analyst may transform those comments into a table containing:

The original comments were unstructured text, but the extracted information
has now been organized into structured fields.

26. Data Extraction

Data extraction is the process of obtaining useful information from a source.

Sources may include:

• Web pages
• PDF documents
• Emails
• Databases
• APIs
• Images
• Applications

The extracted information can then be cleaned and prepared for analysis.

27. Parsing

Parsing means examining data according to its structure so that useful
components can be identified and processed.

For example, a JSON document can be parsed to extract a customer’s name,
age, location and purchase history.

{
    "customer": {
        "name": "Ade",
        "age": 28,
        "location": "Nigeria"
    }
}

An application can parse this information and convert it into fields that
can be used for analysis.

28. Optical Character Recognition (OCR)

Optical Character Recognition (OCR) is a technology used
to extract text from images or scanned documents.

For example, a company may scan thousands of paper forms. OCR can help convert
the text contained in those images into machine-readable information.

After extraction, the resulting information can be cleaned and organized
into structured datasets.

29. Natural Language Processing

Natural Language Processing (NLP) is a field of artificial
intelligence that helps computers process and understand human language.

NLP can be used to analyze:

• Customer reviews
• Emails
• Chat messages
• Survey responses
• Social media comments

One example is sentiment analysis, where a system determines whether text
expresses a positive, negative or neutral opinion.

30. Computer Vision

Computer vision involves techniques that enable computers to interpret
information from images and video.

Applications include:

• Object detection
• Face recognition
• Medical image analysis
• Quality inspection
• Traffic monitoring

Computer vision can turn visual information into useful analytical data.

31. Choosing the Right Storage System

The type of data an organization collects can influence how it should be stored.

For highly structured transactional information, a relational database may
be appropriate.

For flexible documents and JSON records, a document-oriented or NoSQL system
may be useful.

For large collections of mixed-format files, object storage or a data lake
may be more suitable.

The goal is to choose a storage approach that supports the organization’s
requirements for performance, scalability, security and analysis.

32. Real-World Example: Dwallnet Training Center

Imagine Dwallnet Technologies wants to analyze student information.
The organization may collect:

• Student registration records
• Course enrollment records
• Payment information
• Attendance records
• Examination scores
• Student feedback
• Photographs
• Certificates
• WhatsApp messages
• Email communication

The registration and payment tables may be structured data.
Student feedback may be unstructured text.
JSON information from a website application may be semi-structured.
Student photographs are unstructured image data.

A data analyst must understand these differences before deciding how each
dataset should be processed.

33. Why Data Classification Matters

Classifying data correctly helps an analyst determine:

• How data should be stored.
• How it should be cleaned.
• Which tools should be used.
• Which analysis methods are appropriate.
• Which visualizations are suitable.
• How the data should be secured.
• How the information can be integrated with other datasets.

Incorrect classification can lead to inefficient storage, poor analysis and
incorrect conclusions.

34. Practical Exercise

Classify each of the following as structured, semi-structured or unstructured:

1. An SQL customer table
2. A JSON API response
3. A company logo
4. A student Excel spreadsheet
5. A customer voice recording
6. An XML document
7. A training video
8. A PDF report
9. A bank transaction database
10. A social media photograph

For each item, explain why you selected that category.

35. Knowledge Check

1. What is structured data?
2. Give five examples of structured data.
3. What is a schema?
4. What is a primary key?
5. What is a foreign key?
6. What is semi-structured data?
7. Give three examples of semi-structured data.
8. What is unstructured data?
9. Why is unstructured data difficult to analyze?
10. What is metadata?
11. What does OCR mean?
12. What is NLP?
13. What is computer vision?
14. What is the difference between a data warehouse and a data lake?
15. What is the difference between ETL and ELT?

36. Key Terms

• Structured Data:
  Data organized according to a predefined structure.
• Semi-Structured Data:
  Data with some organizational structure but without a rigid relational table format.
• Unstructured Data:
  Data without a predefined tabular structure.
• Schema:
  A definition of how data is organized.
• Primary Key:
  A field that uniquely identifies a record.
• Foreign Key:
  A field used to connect records between related tables.
• Metadata:
  Information that describes other data.
• Parsing:
  Processing data to identify and extract meaningful components.
• OCR:
  Technology used to extract text from images or scanned documents.
• NLP:
  Techniques for processing and analyzing human language.
• Computer Vision:
  Technology for processing and interpreting images and video.
• Data Warehouse:
  A centralized analytical system designed primarily for organized data.
• Data Lake:
  A storage system capable of holding large amounts of data in different formats.

Lesson Summary

Data is not always stored in the same way. Some data fits neatly into rows
and columns, while other information may exist as documents, images, videos,
audio recordings or flexible data structures.

Structured data follows a predefined schema and is commonly
stored in relational databases and spreadsheets.

Semi-structured data has some organization but does not
necessarily follow a rigid table format. JSON and XML are common examples.

Unstructured data does not follow a predefined tabular
structure. Images, videos, audio, documents and free-form text are common
examples.

Modern data analysis increasingly requires analysts to work with all three
categories. Understanding the differences helps analysts select appropriate
storage systems, tools, processing techniques and analytical methods.

In the next lesson, we will study the Data Analysis Workflow
and learn how analysts move from a business question to data collection,
cleaning, analysis, visualization, interpretation and decision making.