• Root Level
• Intermediate Level
• leaf Level.

Clustered index leaf-level pages contain the data in the table whereas Non Clustered index leaf-level pages contain the key value and a pointer to the data row in the clustered index or heap.

Read More

• How to use Resumable Online Index Rebuild Operation?
• Understanding Online Index Rebuild Operation
• SQL Server Interview Questions & Answers on Indexes

If your table does not have SQL Server indexes and you want to access data from the table then database engine will take more time and performance because full table scan will be performed to get that data. However, if your table has an index on column, SQL Server can seek directly on the value and retrieve the rows.

Index Types

There are multiple types of indexes in SQL Server. Below is the list of such indexes.

Clustered Index

• Clustered indexes sort and store the data rows in the table or view based on their key values. There can be only one clustered index per table, because the data rows themselves can be sorted in only one order.
• The only time the data rows in a table are stored in sorted order is when the table contains a clustered index. When a table has a clustered index, the table is called a clustered table. If a table has no clustered index, its data rows are stored in an unordered structure called a heap.

Nonclustered Index

• Nonclustered indexes have the same B-tree structure as clustered indexes.
• The data rows of the underlying table are not sorted and stored in order based on their nonclustered keys.
• The leaf layer of a nonclustered index is made up of index pages instead of data pages.
• You can create upto 999 nonclustered indexes on a table.

Memory-optimized nonclustered indexes

• For memory-optimized nonclustered indexes, memory consumption is a function of the row count and the size of the index key columns

Hash Index

• With a hash index, data is accessed through an in-memory hash table. Hash indexes consume a fixed amount of memory, which is a function of the bucket count.

Unique  Index

• A unique index ensures that the index key contains no duplicate values and therefore every row in the table or view is in some way unique. Uniqueness can be a property of both clustered and nonclustered indexes.

Columnstore Index

• An in-memory columnstore index stores and manages data by using column-based data storage and column-based query processing.
• Columnstore indexes work well for data warehousing workloads that primarily perform bulk loads and read-only queries.
• Use the columnstore index to achieve up to 10x query performance gains over traditional row-oriented storage, and up to 7x data compression over the uncompressed data size.

Index with included columns

• A nonclustered index that is extended to include nonkey columns in addition to the key columns.

Index on computed columns  

• An index on a column that is derived from the value of one or more other columns, or certain deterministic inputs.

Filtered Index

• An optimized nonclustered index, especially suited to cover queries that select from a well-defined subset of data. It uses a filter predicate to index a portion of rows in the table. A well-designed filtered index can improve query performance, reduce index maintenance costs, and reduce index storage costs compared with full-table indexes.

Spatial  Index

• A spatial index provides the ability to perform certain operations more efficiently on spatial objects (spatial data) in a column of the geometry data type. The spatial index reduces the number of objects on which relatively costly spatial operations need to be applied.

XML Index

• A shredded, and persisted, representation of the XML binary large objects (BLOBs) in the xml data type column.

Full-text Index

• A special type of token-based functional index that is built and maintained by the Microsoft Full-Text Engine for SQL Server. It provides efficient support for sophisticated word searches in character string data.

How SQL Server Indexes are used by Query Optimizer

When a query is executed, the query optimizer evaluates each available method for retrieving the data and selects the most efficient method. The method may be a table scan, or may be scanning one or more indexes if they exist. When performing a table scan, the query optimizer reads all the rows in the table, and extracts the rows that meet the criteria of the query. A table scan generates many disk I/O operations and can be resource intensive. However, a table scan could be the most efficient method if, for example, the result set of the query is a high percentage of rows from the table.

When the query optimizer uses an index, it searches the index key columns, finds the storage location of the rows needed by the query and extracts the matching rows from that location. Generally, searching the index is much faster than searching the table because unlike a table, an index frequently contains very few columns per row and the rows are in sorted order.

Your task is to design and create SQL Server indexes that are best suited to your environment so that the query optimizer has a selection of efficient indexes from which to select. SQL Server provides the Database Engine Tuning Advisor to help with the analysis of your database environment and in the selection of appropriate indexes.

I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post Understanding SQL Server Indexes appeared first on Techyaz.com.

Read more about auto growth and auto shrink operation in SQL Server. Both are very important activity you should review to improve your database performance.

• Understanding Database Autogrowth in SQL Server
• Why Should You Always Turn Off Database Auto Shrink Property?

What is Instant File Initialization?

Data and log file initialization takes lot of time if you have to create large database files. This can also increase total completion time of large database restore or large amount of data import to the SQL Server. Microsoft has given a feature called Instant file initialization that allows fast data file allocations for all data file operations. This feature reclaims used disk space without filling that space with zeros. Instead, disk content is overwritten as new data is written to the files. But unfortunately, this feature doesn’t work with log files so they cannot be initialized instantaneously. This is not enterprise feature any SQL Server edition can use it.

As we know DELETE operation deallocates the space and a new allocation can reuse that space which may still have data in it. When we do not zero out the existing space in case of instant file initialization there is a possibility that someone could read data that you thought had been deleted. This is a known security risk of this feature that must be considered while enabling it for your SQL Server Instance. Instant File Initialization is not enabled by default in SQL Server because of this security risk.

Enable Instant File Initialization for SQL Server?

Below are the steps you need to perform to enable Instant File Initialization.

Step 1- Login to server where you want to enable Instant File Initialization. This might be your database server or backup server.
Step 2- Open the Local Security Policy application. You can go to Control Panel->Administrative Tools->Local Security Policy. You can also open it directly by putting secpol.msc into run command prompt.

Step 3- In the left pane, expand Local Policies, and then click User Rights Assignment.

Step 4- In the right pane, double-click Perform volume maintenance tasks.

Step 5- Click Add User or Group and add any user accounts that are used to run SQL Server services. You can get SQL Server service account from SQL Server Configuration Manager or from services.msc console.

Now add account Domain\techyaz to above policy. It would be great if you create a group and add this account to that group and then add that group to given policy so that every time you don’t need to add user account here if SQL Server service account got changed.

Step 6- Once SQL Server service account will be added, it will look like below image. Click Apply, and then close all Local Security Policy dialog boxes.

Step 7- Restart SQL Server services to apply this change.

If you need to enable this feature on Cluster environment, make sure to perform this activity on all cluster nodes. If you are running with multiple SQL Server Instances, add all required security logins/group to this policy.

How to Check If instant file initialization is Enabled Or Not?

Now if you think how to know Instant File Initialization is enabled on your SQL Server Instance then here is the answer. You should check SQL Server logs to get the details about Instant File Initialization. When SQL Server service starts, database engine checks all details of SQL Server configurations and capture them into log file. You can see my error log file after restarting SQL Server service. Instant File Initialization is showing as enabled on this server.

2017-06-16 11:16:49.15 Server Database Instant File Initialization: enabled. For security and performance considerations see the topic ‘Database Instant File Initialization’ in SQL Server Books Online. This is an informational message only. No user action is required.

Advantages

Instant File Initialization improves performance of multiple database activities where large size of data file creation or Autogrow operation will perform. This can vastly speed up these operations as zero-initialization can take a long time for large files. Below are the operations that can take advantage from this feature.

• Data Load/Import
• New Database Creation
• Database Restore
• Autogrow Operation
• Data file size Increase
• Backup file Creation

Limitations

This feature will not work if:

• Your file is a log file.
• The account has not the required privilege.
• It is a sparse file – Database Snapshots.
• Transparent Data Encryption (TDE) is enabled.
• Trace flag 1806 is enabled to disable instant file initialization.

I hope you like this article. Please follow us on our facebook page and on Twitter handle to get latest updates.

Read More: 

• SQL Server Error 5009: One or more files listed in the statement could not be found or could not be initialized
• Fix SQL Server Msg 5144 and 5145 for long or failed autogrow of data and log files
• Manage Transaction Log File during Data Load

The post What is Instant File Initialization and How to enable it on SQL Server Instance? appeared first on Techyaz.com.

Related Articles:

• Why Should You Always Turn Off Database Auto Shrink Property?
• Understand the Best Value for Database File Autogrowth Setting
• What is Checkpoint Process in SQL Server?

Virtual Log File

As we know Transaction Log Files internally divided in to small virtual log files. Virtual Log Files also referred as VLF/VLFs. The size or number of virtual log files cannot be configured by administrators but it can be affected by number of reasons.

One of the reason is creating or extending log files. The size of the virtual files after a log file has been extended is the sum of the size of the existing log and the size of the new file increment. Here I will show you how VLFs will be increase when we add a new log file or increase the size of transaction log file.

Having large no. of virtual log files are not a good sign and can affect the performance of backup, restore and database startups. Large numbers means close to thousand or more than thousand vlfs. You will not see any visible impact if you have approx. few hundred vlfs.  I would suggest you to read Transaction Log File Architecture to understand more about this.

We can reduce the number of virtual log files by either Backup the Transaction Log of the database or by shrinking the Transaction Log as much as possible using DBCC SHRINKFILE.  Make sure to do this exercise during a maintenance window.

How to Find Total Number of Virtual Log Files?

Now, I will explain how to get total number of virtual log files (VLF/VLFs) for each database on your SQL Server instance. Microsoft has introduced new DMF to monitor, alert, and avert potential transaction log issues in SQL Server 2017. Earlier we were using DBCC LOGINFO to get such information but now we can get in depth information to deal with transaction log queries with the help of this DMF.

The new DMF is sys.dm_db_log_info that will returns VLF information of the transaction log files.  If we will specify NULL or DEFAULT value, it will return VLF information of the current database. The built-in function DB_ID can also be specified to get details of a particular database. The sys.dm_db_log_info DMF replaces the DBCC LOGINFO statement from earlier versions.

Below script will show you total no. of virtual log Files for all databases hosted on a SQL Server Instance.

SELECT name, count(d.database_id) as "Total_VLF_Count" from sys.databases sd
cross apply sys.dm_db_log_info(sd.database_id) d
group by name

We can see the total no. of VLFs in each database. Now we can create a new database and check the VLF count. I have created a new database “Techyaz” and see the database file details.

Create database Techyaz
GO
sp_helpdb techyaz

Now again check the total no. of VLFs by running same command which I have executed in step 1. We can see total no. of vlfs are 4 for techyaz database.

Now I will increase the size of the log file and see the impact of vlf count. Run below command to increase the size of log file.

USE master;
GO
ALTER DATABASE Techyaz
MODIFY FILE
(NAME = techyaz_log,
SIZE = 15MB);

Now again see the no. of vlfs for each database and here we can see, total no. of vlfs have been increase to 8 now.

Similarly we can notice the impact of vlf if we add another log file on this database. Run below command to add a log file to the database Techyaz. I kept the size of this log file is 5MB.

ALTER DATABASE Techyaz
ADD LOG FILE 
(
    NAME = techyaz_log3,
    FILENAME = '/var/opt/mssql/data/Techyaz_log3.ldf',
    SIZE = 5MB,
    MAXSIZE = 100MB,
    FILEGROWTH = 5MB
)

Check and verify the newly added log file by running sp_helpdb command.

Check again the total vlf count for this database. Here we can see the total no of vlfs are now 12 in this database. So it is proved that vlf counts are increased if we increase the size of log file as well as we add another log file to the database.

This DMF is also useful in getting filtered output like which database have a specified no of vlf counts and what all databases that have less or more no of vlf counts of a specified value.

Run below command to get the name of databases which has more than 10 virtual log files. You can change the 10 value to 100 or 1000 or 10000. It is up to your requirement.

SELECT name, count(d.database_id) as "Total_VLF_Count" from sys.databases sd
cross apply sys.dm_db_log_info(sd.database_id) d
group by name
having count(d.database_id)>10

We can see only one database has more than 10 vlfs that is techyaz which we just created. Similarly we can check about the db names which has less than 10 VLFs by running same command, just change the condition of greater than to less than.

SELECT name, count(d.database_id) as "Total_VLF_Count" from sys.databases sd
cross apply sys.dm_db_log_info(sd.database_id) d
group by name
having count(d.database_id)<10

Below is the view of all columns we can see of this DMF. This is very useful DMF and we can get all information as given below. I ran this command for database Techyaz. We can get the db id of this database from step 2 that is 6. We can also see one row for each VLF in the output as this database has 12 VLF so there are 12 rows here along with the details of each VLF.

SELECT * from sys.dm_db_log_info(6)

We can focus on two columns i.e. vlf_active which is 1 for first vlf and 0 for all that means it indicates whether VLF is in use or not. 0 means vlf is not in use and 1 means vlf is active so we can see only one vlf is active for this database.

Another is vlf_status, it indicate status of the VLF. Possible values include

0 – vlf is inactive
1 – vlf is initialized but unused
2 – vlf is active.

As we know this DMV is the replacement of DBCC LOGINFO so we can compare the output of both. Now run DBCC LOGINFO as see the details for database Techyaz.

I hope you like this article. Please follow us on our facebook page and on Twitter handle to get latest updates.

Read More:

• SQL Server Architecture Interview Questions & Answers
• Manage Transaction Log File during Data Load
• How to Improve Bulk Data Load Performance?
• Fix SQL Server Error 1105: Could Not Allocate Space for Object in Database because the Filegroup is Full

The post How to Get Total Number of Virtual Log Files inside SQL Server Database Log Files? appeared first on Techyaz.com.

What is Checkpoint Process in SQL Server?

As per book online, a checkpoint creates a known good point from which the SQL Server Database Engine can start applying changes contained in the log during recovery after an unexpected shutdown or crash.

There are lot of operations run in database. If SQL Server performs any modifications to any database pages in memory/buffer cache, SQL Server doesn’t write these modified pages to disk after every change. Rather, SQL Server Database Engine periodically issues a checkpoint on each database and these modified pages and transaction logs in memory will be written to disk as part of checkpoint process.

Checkpoint Types

SQL Server supports four types of checkpoints. Which checkpoint SQL Server database engine will use to write dirty pages to disk is based on their nature of operations and few configuration parameters which we set on instance level as well as database level. Below are the list of checkpoints SQL Server performs.

Automatic Checkpoints

An automatic checkpoint occurs each time the number of log records reaches the number the Database Engine estimates it can process during the time specified in the recovery interval server configuration option. You can set the value of recovery interval using below system stored procedure.

sp_configure 'recovery interval','seconds'

Recovery interval is maximum time that a given server instance should use to recover a database during a system restart. Database Engine estimates the maximum number of log records it can process within the recovery interval. When a database using automatic checkpoints reaches this maximum number of log records, the Database Engine issues an checkpoint on the database.

The frequency depends on the value of recovery interval option of server configuration. The time interval between automatic checkpoints can be highly variable. A database with a substantial transaction workload will have more frequent checkpoints than a database used primarily for read-only operations.

Database Engine generates automatic checkpoints for every database if you have not defined any value for target recovery time for a database. If you define user value for target recovery time for any database, database engine will not use automatic checkpoint and generate indirect checkpoint which I will discuss in next section.

Under the simple recovery model, an automatic checkpoint is also queued if the log becomes 70 percent full.

After a system crash, the length of time required to recover a given database depends largely on the amount of random I/O needed to redo pages that were dirty at the time of the crash. This means that the recovery interval setting is unreliable. It cannot determine an accurate recovery duration. Furthermore, when an automatic checkpoint is in progress, the general I/O activity for data increases significantly and quite unpredictably.

For an online transaction processing (OLTP) system using short transactions, recovery interval is the primary factor determining recovery time. However, the recovery interval option does not affect the time required to undo a long-running transaction. Recovery of a database with a long-running transaction can take much longer than the specified in the recovery interval option.

Typically, the default values provides optimal recovery performance. However, changing the recovery interval might improve performance in the following circumstances:

1. If recovery routinely takes significantly longer than 1 minute when long-running transactions are not being rolled back.
2. If you notice that frequent checkpoints are impairing performance on a database.

If you decide to increase the recovery interval setting, we recommend increasing it gradually by small increments and evaluating the effect of each incremental increase on recovery performance. This approach is important because as the recovery interval setting increases, database recovery takes that many times longer to complete. For example, if you change recovery interval 10 minutes, recovery takes approximately 10 times longer to complete than when recovery interval is set to 1 minute.

Indirect Checkpoints

This checkpoint was introduced in SQL Server 2012. Indirect checkpoints provide a configurable database-level alternative to automatic checkpoints. In the event of a system crash, indirect checkpoints provide potentially faster, more predictable recovery time than automatic checkpoints. We can set the TARGET_RECOVERY_TIME in Indirect checkpoint for any database by executing below ALTER DATABASE statement.

--Change DBNAME with your database name.
ALTER DATABASE DBNAME SET TARGET_RECOVERY_TIME =target_recovery_time {SECONDS | MINUTES}

The difference between Automatic and Indirect checkpoint is, recovery interval configuration option uses the number of transactions to determine the recovery time as opposed to indirect checkpoints which makes use of number of dirty pages.

When indirect checkpoints are enabled on a database receiving a large number of DML operations, the background writer can start aggressively flushing dirty buffers to disk to ensure that the time required to perform recovery is within the target recovery time set of the database. This can cause additional I/O activity on certain systems which can contribute to a performance bottleneck if the disk subsystem is operating above or nearing the I/O threshold.

1. Indirect checkpoints enable you to reliably control database recovery time by factoring in the cost of random I/O during REDO.
2. Indirect checkpoints reduce checkpoint-related I/O spiking by continually writing dirty pages to disk in the background. However, an online transactional workload on a database configured for indirect checkpoints can experience performance degradation. This is because the background writer used by indirect checkpoint sometimes increases the total write load for a server instance.

Indirect checkpoint is the default behavior for new databases created in SQL Server 2016. Databases which were upgraded in place or restored from a previous version of SQL Server will use the previous automatic checkpoint behavior unless explicitly altered to use indirect checkpoint.

Manual Checkpoints

Manual checkpoint issued when you execute a T-SQL CHECKPOINT command for a database. By default, manual checkpoints run to completion. Throttling works the same way as for automatic checkpoints.

Internal Checkpoints

Internal Checkpoints are generated by various server components to guarantee that disk images match the current state of the log. Internal checkpoint are generated in response to the following events:

1. Database files have been added or removed by using ALTER DATABASE.
2. A database backup is taken.
3. A database snapshot is created, whether explicitly or internally for DBCC CHECK.
4. An activity requiring a database shutdown is performed. For example, AUTO_CLOSE is ON and the last user connection to the database is closed, or a database option change is made that requires a restart of the database.
5. An instance of SQL Server is stopped by stopping the SQL Server (MSSQLSERVER) service. Either action causes a checkpoint in each database in the instance of SQL Server.
6. Bringing a SQL Server failover cluster instance (FCI) offline.

Checkpoint generated by SQL Server in different scenario

Now one question might come to your mind that how SQL Server decide which checkpoint should be used between Automatic and Indirect checkpoint if a database has target recovery time value is set to use indirect checkpoint. Below are the configuration values based on which SQL Server considers to generate either automatic checkpoint or indirect checkpoint?

1. Automatic checkpoints whose target recovery interval is 1 minute will be generated if ‘recovery interval’ value in sp_configure command and TARGET_RECOVERY_TIME value in ALTER DATABASE command is set to 0.
2. Automatic checkpoints whose target recovery interval is specified by the user defined setting of the sp_ configure recovery interval option if TARGET_RECOVERY_TIME value in ALTER DATABASE command is set to 0 and if ‘recovery interval’ value in sp_configure command is non zero.
3. Indirect checkpoints when target recovery time is determined by the TARGET_RECOVERY_TIME value in ALTER DATABASE command.

You can follow attached link to understand SQL Server Transaction Log Architecture which is also related to checkpoints topic. I hope you like this article. Please follow us on our facebook page and on Twitter handle to get latest updates.

Read More:

• Manage Transaction Log File during Data Load
• Why Should You Always Turn Off Database Auto Shrink Property?
• understand the best value for database file autogrowth
• SQL Server Architecture Interview Questions & Answers

The post What is Checkpoint Process in SQL Server? appeared first on Techyaz.com.

Related Articles:

• Manage Transaction Log File during Data Load
• Why Should You Always Turn Off Database Auto Shrink Property?
• What is the best value for database file autogrowth?
• What is the Suitable Value for Fill Factor?

Logical Architecture

To understand SQL Server Transaction Log File Architecture, we need to start with basics. As we know transaction log file is mandatory database file for a database. Every database must have at least one transaction log file. SQL Server uses transaction log file to capture log records that guarantee the data integrity of the database and for data recovery.

Logically, SQL Server transaction log is a string of log records. Each log record is identified by a log sequence number (LSN). Each new log record is written with the incremental LSN that means a new log record will be assigned a LSN that is higher than the LSN of the record before it. Log records are stored in a serial sequence as they are created.

Operations that recorded in the Transaction Log File

There are many types of operations recorded in Tlog files. These operations include:

1. The start and end of each transaction.
2. Every data modification (insert, update, or delete). This includes changes by system stored procedures or data definition language (DDL) statements to any table, including system tables.
3. Every extent and page allocation or deallocation.
4. Creating or dropping a table or index.
5. Rollback operations. Each transaction reserves space on the transaction log to make sure that enough log space exists to support a rollback that is caused by either an explicit rollback statement or if an error is encountered. This reserved space is freed when the transaction is completed.

The section of the log file from the first log record that must be present for a successful database-wide rollback to the last-written log record is called the active part of the log, or the active log. This is the section of the log required to do a full recovery of the database. You can not truncate any part of active log. 

Physical Architecture

As we know, a series of log records stores in transaction log file. These log files store physically on OS drive/disk along with SQL Server data files. Each transaction log file internally divided into a number of virtual log files. Virtual log files have no fixed size, and there is no fixed number of virtual log files for a physical log file.  The size or number of virtual log files cannot be configured by administrators. You can get total count of virtual log files or vlfs in a SQL Server database log file/s in attached link. Here, i have explained in detail about how to track virtual log files.

The Database Engine chooses the size of the virtual log files dynamically while it is creating or extending log files. The size of the virtual files after a log file has been extended is the sum of the size of the existing log and the size of the new file increment.

If we define small Tlog file size along with small growth increment then it will cause performance issue to your system because whenever log file needs more room, it will increase log file size frequently due to small size of growth increment so it will create lot of virtual log files as well. This will affect performance of backup and restore operation along with database startup performance.

Microsoft recommends that we should assign log files a size value close to the final size required, and also have a relatively large growth increment value.

How Transactions Log File works?

The transaction log is a wrap-around file. Let us demonstrate it using a scenario given in Microsoft Book Online.

Consider a database with one physical log file divided into four virtual log files. When the database is created, the logical log file begins at the start of the physical log file. New log records are added at the end of the logical log and expand toward the end of the physical log. Log truncation frees any virtual logs whose records all appear in front of the minimum recovery log sequence number (MinLSN). The MinLSN is the log sequence number of the oldest log record that is required for a successful database-wide rollback.

When the end of the logical log reaches the end of the physical log file, the new log records wrap around to the start of the physical log file.

This cycle repeats endlessly, as long as the end of the logical log never reaches the beginning of the logical log. If the old log records are truncated frequently enough to always leave sufficient room for all the new log records created through the next checkpoint, the log never fills. However, if the end of the logical log does reach the start of the logical log, one of two things occurs:

1. If the FILEGROWTH setting is enabled for the log and space is available on the disk, the file is extended by the amount specified in growth_increment and the new log records are added to the extension.
2. If the FILEGROWTH setting is not enabled, or the disk that is holding the log file has less free space than the amount specified in growth_increment, a 9002 error is generated.

If the log contains multiple physical log files, the logical log will move through all the physical log files before it wraps back to the start of the first physical tlog file.

You can follow attached link to understand SQL Server data files, Pages & Extents.  I hope you like this article. Here i have described SQL Server Transaction Log File Architecture that will help you to understand SQL Server database system. Please follow us on our facebook page and on Twitter handle to get latest updates.

Read More:

• Understanding Database Shrink Operation and Why Should We Avoid It?
• How to Avoid Page Split in SQL Server?
• Fix SQl Server Error 5144 and 5145 for long and failed Autogrow of data and log file

The post Understanding SQL Server Transaction Log File Architecture appeared first on Techyaz.com.

Related Articles:

• Manage Transaction Log File during Data Load
• Why Should You Always Turn Off Database Auto Shrink Property?
• What is the Best Value for Database File Autogrowth?
• SQL Server Architecture Interview Questions & Answers

SQL Server Data Files

When we create any SQL Server database, few operating system files are created and placed in to their respective directories. These files are known as database files. SQL Server databases have three types of files to handle data and log separately.

• Primary data file
• Secondary data files
• Log files

Every database has only one primary data file but none to multiple secondary data files whereas same database has minimum one log file and can have multiple log files depending on the requirements. Log files hold all the log information that is used to recover the database whereas all data files keeps or stores the actual data of the database.

The recommended file extension for primary, secondary data file and log file are .mdf, .ndf and .ldf respectively although it is not mandatory to keep same extension names but it helps you identify the different kinds of files and their use.

Each database file has two names one is logical file name another one is physical file name. The logical file name is the name used to refer to the physical file in all Transact-SQL statements whereas physical file name is the name that appear on the OS directory path.

Pages & Extents

We read in above section that every database has data files to store actual data. Data stores inside these data files in form of Pages so Page is the fundamental unit of data storage in SQL Server but Log files do not store any data, they contain a series of log records so log files do not have any pages.

The disk space allocated to a data file (.mdf or .ndf) in a database is logically divided into pages numbered contiguously from 0 to n. Pages in a SQL Server data file are numbered sequentially, starting with zero (0) for the first page in the file. Each file in a database has a unique file ID number. To uniquely identify a page in a database, both the file ID and the page number are required.

The size of page is 8 KB. Each page begins with 96-byte header as shown in above image that is used to store system information about the page. This information includes the page number, page type, the amount of free space on the page, and the allocation unit ID of the object that owns the page.

Data rows are put on the page serially, starting immediately after the header. A row offset table starts at the end of the page, and each row offset table contains one entry for each row on the page. Each entry records how far the first byte of the row is from the start of the page. The entries in the row offset table are in reverse sequence from the sequence of the rows on the page.

Extents are a collection of eight physically contiguous pages and are used to efficiently manage the pages. All pages are stored in extents.

The first page in each file is a file header page that contains information about the attributes of the file. Several of the other pages at the start of the file also contain system information, such as allocation maps. Below are the list of page types used in the data files of a SQL Server database.

1. Data: Data Page stores data rows with all data, except large object data types like text, ntext, image, nvarchar(max), varchar(max), varbinary(max), and xml data, when text in row is set to ON.
2. Index: Stores Index data.
3. Text/Image: Stores large object data types like text, ntext, image, nvarchar(max), varchar(max), varbinary(max), and xml data
4. Global Allocation Map: GAM pages record what extents have been allocated. Each GAM covers 64,000 extents, or almost 4 GB of data. The GAM has one bit for each extent in the interval it covers. If the bit is 1, the extent is free; if the bit is 0, the extent is allocated.
5. Shared Global Allocation Map: SGAM pages record which extents are currently being used as mixed extents and also have at least one unused page. Each SGAM covers 64,000 extents, or almost 4 GB of data.

The SGAM has one bit for each extent in the interval it covers. If the bit is 1, the extent is being used as a mixed     extent and has a free page. If the bit is 0, the extent is not used as a mixed extent, or it is a mixed extent and all     its pages are being used.

6. Page Free Space: Page Free Space (PFS) pages record the allocation status of each page, whether an individual page has been allocated, and the amount of free space on each page. The PFS has one byte for each page, recording whether the page is allocated, and if so, whether it is empty, 1 to 50 percent full, 51 to 80 percent full, 81 to 95 percent full, or 96 to 100 percent full.

After an extent has been allocated to an object, the Database Engine uses the PFS pages to record which pages     in the extent are allocated or free. This information is used when the Database Engine has to allocate a new           page.

7. Index Allocation Map: Information about extents used by a table or index per allocation unit. When the SQL Server Database Engine has to insert a new row and no space is available in the current page, it uses the IAM and PFS pages to find a page to allocate, or, for a heap or a Text/Image page, a page with sufficient space to hold the row. The Database Engine uses the IAM pages to find the extents allocated to the allocation unit.
8. Bulk Changed Map: Information about extents modified by bulk operations since the last BACKUP LOG statement per allocation unit. This tracks the extents that have been modified by bulk logged operations since the last BACKUP LOG statement.

If the bit for an extent is 1, the extent has been modified by a bulk logged operation after the last BACKUP LOG   statement. If the bit is 0, the extent has not been modified by bulk logged operations.

Although BCM pages appear in all databases, they are only relevant when the database is using the bulk-logged recovery model. In this recovery model, when a BACKUP LOG is performed, the backup process scans the             BCMs for extents that have been modified. It then includes those extents in the log backup. This lets the bulk       logged operations be recovered if the database is restored from a database backup and a sequence of   transaction  log backups.

 BCM pages are not relevant in a database that is using the simple recovery model, because no bulk logged  operations are logged. They are not relevant in a database that is using the full recovery model, because that  recovery model treats bulk logged operations as fully logged operations.

9. Differential Changed Map: Information about extents that have changed since the last BACKUP DATABASE statement per allocation unit. This tracks the extents that have changed since the last BACKUP DATABASE statement.

  If the bit for an extent is 1, the extent has been modified since the last BACKUP DATABASE statement. If the bit is 0, the extent has not been modified.

Differential backups read just the DCM pages to determine which extents have been modified. This greatly reduces the number of pages that a differential backup must scan. The length of time that a differential backup runs is proportional to the number of extents modified since the last BACKUP DATABASE statement and not the overall size of the database.

Sequence of Pages in Data file

A PFS page is the first page after the file header page in a data file (page number 1). This is followed by a GAM page (page number 2), and then an SGAM page (page 3). There is a PFS page approximately 8,000 pages in size after the first PFS page. There is another GAM page 64,000 extents after the first GAM page on page 2, and another SGAM page 64,000 extents after the first SGAM page on page 3. The DCM and BCM pages are located behind the GAM and SGAM pages in a physical file and interval between DCM pages and BCM pages is the same as the interval between GAM and SGAM page, 64,000 extents.

I hope this article helps you to understand data files, pages and extents. The source of this article is MSDN Book Online.  If you like this article, you can follow us on our facebook page and on Twitter handle to get latest updates.

The post Understanding SQL Server Data Files, Pages & Extents appeared first on Techyaz.com.