SQLServer Error: 802, There is insufficient memory available in the buffer pool. [SQLSTATE 42000]

SQL Server Error 802 – Insufficient Memory Available in the Buffer Pool

The main reason behind getting SQL Server error 802 is your SQL Server instance is under memory pressure and there is not enough memory available to allocate further to buffer pool or buffer pool is restricted to not use memory beyond a certain limit. This might be because of various reasons as i have stated in below section. One of the very easy and straight forward approach to fix this error is to increase more memory to SQL Server system and modify max server memory in SQL Server accordingly.

Analysis & Solution

I would suggest to go deeper in this issue before taking any decision to allocate more memory to the system where SQL Server is running. There could be multiple temporary reasons as well which can cause SQL Server to generate below error log:

SQL Server Error: 802, There is insufficient memory available in the buffer pool. [SQLSTATE 42000]

Let’s discuss this issue further and ensure to not forget below points before taking any decisions.

Please comment us about your issue experience and let me know how you have fixed your issue.

Related Articles:

• How to Fix SQL Server Error 701: There is insufficient system memory to run this query
• Understanding Hybrid Buffer Pool in SQL Server
• SQL Server Interview Questions & Answers

The post Fix SQL Server Error 802: There is insufficient memory available in the buffer pool. appeared first on Techyaz.com.

I have already explained top performance counters to detect Memory and Disk IO bottlenecks in SQL Server in my last articles. Have a look at them (attached below) to get Memory and Disk IO related performance counters and their ideal values for healthy system.

• Top 10 Performance Counters to Identify SQL Server Memory Pressure
• Identify Disk Bottleneck in SQL Server using Perfmon Disk Counters

Generally, CPU bottleneck arises because of either your system has not enough CPUs to process current workload or your transactions or applications are poorly designed that causes the high CPU. To deal with SQL Server CPU pressure, we need to either find the high CPU intensive queries and optimize them or increase the total CPU counts of the database server to handle current workload. There can be multiple reasons for high CPU usage like high recompilations of SQL Queries, OS issues, Query design issues: Sorting Joins, Poor Indexes, Condition Clauses etc. Here, I am going to explain about top perfmon counters to identify SQL Server CPU bottleneck.

Perfmon Counters for CPU Usage

Here, I am going to describe about perfmon counters for CPU usage in SQL Server. Ideally, first we should start with looking at CPU usage in Windows Task Manager. If the CPU utilization is continuously going beyond 85%, it means your system is facing CPU bottleneck. Find the list of performance counters to identify SQL Server CPU bottleneck.

Processor: % Processor Time 

One of the very useful CPU perfmon counter for SQL Server is Processor: % Processor Time. This counter displays the percentage of time CPU spends executing a thread that is not idle. If your system is frequently reaching beyond 85% then it means your system is facing CPU bottleneck. The acceptable value for this counter should be less than 50%.

Processor: % User Time

Windows operating system has two modes of processing for any process. One is Kernel mode another one is user mode. This counter shows the percentage of time a process spends executing user processes like SQL Server. If this counter is high it means CPU pressure is because of SQL Server or because of user applications. You should focus to retrieve CPU intensive queries to get the culprit transactions that are causing high CPU pressure. There is no idle value for this counter. You should benchmark each system based on your workload.

Processor: % Privilege Time

This counter shows the percentage of time the processor spends on execution of Microsoft Windows kernel commands, such as processing of SQL Server I/O requests. This counter captures the information about kernel mode and indicates towards poor performance of disk subsystems. Monitor DISK IO related performance counters to identify DISK IO bottleneck and if possible, consider installing a faster or more efficient disk subsystem.

Processor: % Interrupt Time

This counter Processor: % Interrupt Time tells about the time a processor spends receiving and servicing hardware interruptions during specific sample intervals. This counter indicates a possible hardware issue if the value is greater than 15 percent.

System: Processor Queue Length

This is an important perfmon counter to detect CPU bottleneck. This counter gives information about the number of threads waiting for processor time. If value of this counter is greater than zero then this is a sign of CPU bottleneck. We can further check Compilations/sec and Re-Compilations/sec. A high number of compilations and recompilations usually indicates poor reuse of the query plans. This can be fixed by optimizing your queries and stored procedures.

System: Calls/sec

Any system would have either software calls or application related calls or hardware related calls. This counter System: Calls/sec metric shows the number of software calls to the operating system service routines per second. Similarly, another CPU perform counter Processor: Interrupts/Sec shows the number of hardware calls to the processor. Both perfmon counters can be used to determine the source of the performance issues. The acceptable values for System: Calls/sec are under 20,000. Higher values might indicate excessive operating system activity caused by software applications.

Context Switches/sec

Context Switches occurs when kernel switches processor from one thread to another. The acceptable Context Switches/sec value should be less than 2,000 per processor. If you are observing constant high values of Context Switches/sec and Processor: % Processor Time on your system then this would be a clear sign of processor bottlenecks. Permanent solution to reduce CPU bottleneck is to change the processor to a more powerful one, or add an additional to existing ones.

You might have seen high Context Switches/sec values for systems that are running with multiple SQL Server instances or a system that is hosted other applications along with SQL Server.

Here I have described top perfmon counters for CPU usage on your database server. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

Read More:

• Disk and Performance Impact of running Online Index Rebuild Operation
• Understanding Database Shrink Operation and Why should you Avoid it?
• Fix Error 701: There is Insufficient System Memory to Run this Query
• How to Improve Bulk Data Load Performance?

The post Top Perfmon Counters to Identify SQL Server CPU Bottleneck appeared first on Techyaz.com.

• Fix SQL Server Error 701: There is Insufficient System Memory to Run this Query

We can use Windows PerfMon utility to gather current values of all given performance counters. Just type perfmon in run prompt to launch perfmon tool to gather memory related performance counters. Although, Microsoft has also given a DMV related to performance counters that is very useful in collecting data and detecting CPU, IO or memory bottleneck in SQL Server. You can run below command to get all performance counters and their respective values using DMV sys.dm_os_performance_counters. You can also use this DMV to get performance counter values to identify memory pressure in SQL Server.

Select * from sys.dm_os_performance_counters

By default, SQL Server changes its memory requirements dynamically, on the basis of available system resources. I have explained SQL Server Memory Management and how SQL Server allocated memory to their transactions in below attached article.

• SQL Server Memory Allocation & Role of MIN/MAX Server Memory Configuration

Performance Counters to Identify SQL Server Memory Pressure

Let’s discuss top perfmon counters for memory usages to identify its bottleneck. Below is the list of top 10 performance counters to identify SQL Server Memory pressure. If you want to read CPU and Disk IO related perfmon counters then i would suggest you to read below attached articles.

• Top Perfmon Counters to Identify SQL Server CPU Bottleneck
• Identify Disk Bottleneck in SQL Server using Perfmon Disk Counters

Memory: Available Bytes

First thing you should look at in case of memory bottleneck in SQL Server is the available memory that can be used by processes. You can get this value by looking at the counter Available Bytes under object Memory. Low value of this counter shows the memory shortage on this server.  Now, you should focus behind the reason of this memory shortage. There might be possibility that some of your applications running on your server are not releasing memory or your system has not enough memory to perform current workload that leads to the memory bottleneck in SQL Server.

Memory: Pages/sec

This counter indicates the total number of pages that are either read from disk or written to the disk due to page fault. When a process requests a data page that is not in memory then it will be retrieved from disk to memory. This process is known is page fault. High value of this counter could indicate the excessive paging. We should further monitor Memory: Page Faults/sec counter to go indepth information. The average Pages/sec value should be below 50. A high value of Pages/sec indicates intensive memory activities like read from and write to disk. It means transactions are getting enough memory that is why they are using disk to process their requests.

Memory: Page Faults/sec

High value of counter Memory: Pages/sec could indicate the excessive paging. Next, we should monitor the Memory: Page Faults/sec counter to make sure that the disk activity is not caused by paging. If you want to determine whether SQL Server or another process is causing excessive paging then we should monitor the Process: Page Faults/sec counter for the SQL Server process instance. This will give an exact picture whether SQL Server is causing this paging issue or not.

Process: Working Set

This counter shows the amount of memory a process can use. This counter value shows whether you have set optimum value in min server memory and max server memory configuration or not. If this value shows constantly below min server memory and max server memory then you have configured too much memory for your SQL Server.

SQL Server: Buffer Manager: Buffer Cache Hit Ratio

This is very important counter and this counter shows the number of percentage that has been satisfied using data cache. Suppose the value of buffer cache hit ratio for your server is 94 it means 94% of total requests for data have been processed using data cache.

The ideal value for this counter is more than 90. Higher the value better the memory health.

SQL Server: Buffer Manager: Page Life Expectancy

Page Life Expectancy is very important counter to measure memory pressure for your system. THis counter indicates the average number of seconds a data page to remain in cache. Ideally, we have read multiple blogs to consider 300 or more as acceptable value but this value varies depending on your system configuration.

The ideal value is more than 300 but always compare this value with your system configuration. Higher the better concept is also applying here.

SQL Server: Buffer Manager: Free List Stalls/sec

This counter also shows a sign of memory pressure or insufficient memory on your server. This displays the number of requests per second where data requests stall because there are no buffers available. Any value greater than two indicates that SQL Server needs more memory.

SQL Server: Buffer Manager: Lazy Writes/sec

This counter tells about total number of Lazy writer performed in a second.  It means total number of times Lazy Writer process moves dirty pages from the buffer to disk. A lower number is better than a higher number. If value of this counter shows higher value continuously more than 20, then we need to add more memory to the system or we should think about optimizing memory intensive operations.

SQL Server: Buffer Manager: Checkpoint Pages/sec

This counter monitors the number of dirty pages per second, which are flushed to disk when SQL Server invokes the checkpoint process. High values for this counter may indicate memory pressure or insufficient memory.

SQL Server: Memory Manager: Memory Grants Pending

This counter shows the number of processes waiting for memory grant per second. Any value higher than zero indicates a lack of memory. We can also get this information by using a DMV sys.dm_exec_query_memory_grants. Those processes who are waiting to be granted the required memory will show you waiting with RESOURCE_SEMAPHORE wait type. I have explained about resource semaphore wait type in below attached article.

• Fix RESOURCE_SEMAPHORE Wait Type in SQL Server

Here, I have described top 10 performance counters to identify SQL Server memory pressure on your database server. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

Read More:

• Fix Error 833: SQL Server has encountered 32 occurrences of I/O requests taking longer than 15 seconds to complete.
• Disk and Performance Impact of running Online Index Rebuild Operation
• Understanding ASYNC_IO_COMPLETION Wait Type

The post Top 10 Performance Counters to Identify SQL Server Memory Pressure appeared first on Techyaz.com.

Error 701, Severity: 17, State: 193.
There is insufficient system memory to run this query.

Or

Error 802, Severity: 17, State: 20
There is insufficient memory available in the buffer pool.

Error 701 or Error 802: Root Cause

If SQL Server is running out of memory and has no more memory available to allocate to its transactions then it will generate SQL Server memory error 701 or error 802. SQL Server Memory pressure or insufficient memory issue can be raised due to huge data load or other memory intensive operations.

We should check DBCC MEMORYSTATUS output to get latest snapshot of current memory status of the SQL Server. This command is very useful in troubleshooting memory issues in SQL Server. You might even have failed to run this DBCC command because system does not have enough memory. You need to reduce some memory pressure by stopping or killing some processes and then try to run this command to gather in depth information about SQL Server memory state.

The transaction that failed due to memory pressure and generates error 701 or error 802 might be or might not be the cause of this error. We should analyze the trends to reach on any conclusion.

Solution

There are mainly two reasons behind getting SQL Server memory issues for most of the SQL Server systems. Either your server has less memory to handle the current day to day workload or you are running some of the adhoc transactions that will require more memory for their execution like huge bulk data load, complex reports etc. If your SQL Server is running with insufficient memory during day to day activities then you should seriously evaluate the total workload and propose optimum memory size that can handle your current workload.

If you are planning to run any heavy ad hoc transaction then you should temporary increase the memory that can handle the transaction smoothly or you should run such transactions on a server that are built to handle such loads. You should also consider to reduce the batch size that is optimum for your system.

SQL Server is memory intensive application. We need to plan memory sizing for a server after proper evaluation of server state and overall load that will run on that SQL Server instance. But if you are facing memory issue on one of your SQL Server then first we need to troubleshoot it and find the root cause of this memory pressure. Here I am giving you step by step method to find the actual culprit that are causing the memory issue and possible solutions that can reduce some memory overheads.

First Make sure your SQL Server instance is running on dedicated server and not with other applications. If other applications are also using your database server then make sure they are not eating most of the memory. And, If most of the system memory is taken by other applications then you need to seriously think about migrating those application from this database server.

If you cannot migrate these applications to another server then workaround for such issues are to allocate the maximum memory value to your SQL Server instance. You can do it either by using sp_configure or using GUI. Read attached article to change max memory value for your SQL Server instance.

• Understanding SQL Server Memory Allocation and Role of MIN/MAX Server Memory Configuration

Next thing you should check is the server memory configuration. Check below SQL Server memory configuration parameters:

• max server memory
• min server memory
• min memory per query

Notice any unusual settings and fix them as necessary. All above three options have been covered in detain in above attached article.

Now, check the workload in terms of number of concurrent sessions, currently executing queries along with their memory grants. You should also focus on SQL Server transactions that are seeking huge memory values. Look at the transactions and gather a report for high memory intensive operations. Evaluate them and if possible optimize them so that they can take minimum memory.

• Understanding Memory Wait type Resource_SEMAPHORE and how to fix it?

You can also run below DBCC commands to free several SQL Server memory caches. This will also help you and make some relief for your current running transactions but in longer run it will not help if your system will need more memory to proceed the transactions.

• DBCC FREESYSTEMCACHE
• DBCC FREESESSIONCACHE
• DBCC FREEPROCCACHE

Another effective way to look in to memory status is by getting the output of DBCC MEMORYSTATUS command. You would get a good starting point to look in to culprit memory consumers.

We can also start collecting performance monitor counters for memory. Perfmon counters like the value of SQL Server: Buffer Manager\Buffer Cache Hit Ratio, SQL Server: Memory Manager and Page Life Expectancy will be very helpful in identifying memory pressure on SQL Server system. Analyze the trend for these counters and then decide whether issue is coming during some specific operations or system is continuously facing memory pressure.

Based on these analyses you can choose to either increase the system memory, increase the SQL Server memory allocation or reschedule some of the memory specific transactions to the off hours when load is minimum on the system. You can also reduce the batch size of the transaction to reduce the memory pressure. This will be certainly helpful in troubleshooting memory issues.

It is not necessary that your transactions are poorly designed always that is why SQL Server is taking lot of memory to process it. Sometimes even our systems on which SQL Server runs might have problem that end up with memory pressure. So you should consider all these points while fixing any performance issue.
I hope you like this article. Please follow us on our Facebook page and Twitter handle to get latest updates.

Read More:

• 4 Usages of DBCC SQLPERF in SQL Server
• Manage Transaction Log File during Bulk Data Load
• How to Improve Bulk Data Load Performance?
• Overview of SQL Server Performance Tuning
• Identify Disk Bottleneck in SQL Server using Perfmon Disk Counters

The post SQL Server Error 701: There is Insufficient System Memory to Run this Query appeared first on Techyaz.com.

Server Level Configuration Check

To deal with SQL Server Performance Tuning, it is required to review and check all the server level configuration settings of the SQL Server on periodic basis. Some of these settings may vary depending on the workload and criticality of the application. You can have a look at all the configurational settings by running below command.

sp_configure'show advanced options',1
GO
RECONFIGURE
GO
sp_configure

Index Analysis

The role of indexes in SQL Server performance tuning is very crucial. To achieve better Server performance during data retrieval, it is important to create useful indexes. Proper indexing helps in executing tasks with minimal disk Input\ Output operations and fewer system resources.

To implement this approach, it is important to understand how indexes are used by ‘Query Processor’ for executing quick data search operation. Even excessive indexes or poor indexes can harm your database performance so better to analyze the data and query before creating any indexes on your table. Read below articles to understand more on SQL Server Indexes.

• Understanding SQL Server Indexes and difference between Clustered and Nonclustered Indexes

SQL Wait Stat Analysis

“Wait Stat Analysis” is an alternate way to identify the symptoms that cause impact on the Server performance. SQL Server constantly keeps track of the reasons for which execution tasks wait or gets delayed; so to detect the reason, one can refer to the Server logs. You can read more about SQL Server Wait types in below attached articles.

• Understanding SQL Server Wait Types and How to get wait type information?
• How to Reduce PREEMPTIVE_OS_WRITEFILEGATHER Wait type in SQL Server?
• Should We Clear Wait Stats Data from DMV sys.dm_os_wait_stats?
• Understanding Resource_Semaphore wait type
• How to Fix BACKUPBUFFER Wait Type?

TempDB Space Review

Another way for optimizing Server performance tuning is through proper analysis of the usage as well as empty and occupied space statistics of TempDB database. Make sure that you TempDB database files are placed separately on different drive. If you place user database files and TempDB database files on same drive you might face high disk IO that will cause performance degradation.

Index Fragment\ Defragment Operation

This approach can better be implemented through analyzing the indexes to detect the extent of fragmentation and to identify which indexes and databases need to be defragmented for ensuring the smooth performance of these components. You can run SQL Server dashboard report to check which indexes are highly fragmented and need to rebuild or reorganize to improve the performance. Read below articles to understand Index rebuild and Reorganize operation in SQL Server.

• Difference between Index Rebuild and Index Reorganize Operation
• How to use Resumable Online Index Rebuild Operation?
• Disk and Performance Impact of running Online Index Rebuild Operation

Application/Program Logs

By referring to the “Windows Event Log” or “SQL Server Error Log”, the actual cause for the inappropriate functionality of either database or SQL Server can be detected. This could help in identifying and rectifying the existing errors that lead to files inaccessibility.

Backup & Recovery Parameters

For better performance tuning, analyze the backup and recovery settings to ensure if they are implemented appropriately. So that the backup files can be restored successfully in case of any unexpected circumstances.

Review Database Files

Review entire database files including primary, secondary as well as log files (MDF, NDF and LDF respectively) to analyze and detect the actual cause that is responsible for the inappropriate functionality of database. Make a practice to keep data and log files on separate drives to reduce the disk IO pressure. You can also analize which data or log file is having highest IO operating, this way you can plan to place that file in a faster disk.

• Identify Disk Bottleneck in SQL Server using Perfmon Disk Counters
• Understanding Data Files, Pages & Extents

Blocking and Deadlock Detection

Detecting Deadlocks is not a major task but often requires proper understanding and hands-on experience on the concept to implement it in an accurate manner. To implement the task, it is important to understand the locking mechanism along with its resolution. You can also identify if there is poor application design or coding by analyzing blocking and deadlock occurrence. If it occurs frequently, you can seriously think about optimizing your application or T-SQL coding. Identifying blocking issues will also help you in SQL Server performance tuning.

Hardware Components

Regular examination test over the various components of the SQL Server machine is required to ensure smooth functioning of all the elements as well as to prevent the adverse circumstances that can take place due to major hardware breakdown.

DBCC Implementation

It is very important to understand the nature of Database Console Commands as some of them are very critical and sometimes requires additional attention while execution. Some of these commands include DBCC SRHINKDATABASE, DBCC FREEPROCCACHE, DBCC SHRINKFILE, DBCC REINDEX and DBCC DROPCLEANBUFFER along with some stored procedures such as SP_UPDATESTATS.

• 4 Usages of DBCC SQLPERF in SQL Server

Conclusion

So, these are some of the major mechanisms of SQL Server Performance Tuning concept and when implemented with proper consideration and attention, help in optimizing the performance of the Server. By executing these actions on regular instance of time; the performance issues with the Server and the incorporated databases can be resolved.

The post Overview on SQL Server Performance Tuning appeared first on Techyaz.com.

What is PREEMPTIVE_OS_WRITEFILEGATHER?

As per MSDN, PREEMPTIVE_OS_WRITEFILEGATHER is a wait type that occurs while SQL Server is waiting for the Operating System to complete portions of write operations. This wait is commonly seen in databases with auto-growth not configured appropriately or during large size allocation to database files. If you are getting this wait type frequently then you should investigate your SQL Server instance. You can also observe this wait type along with BACKUPBUFFER  and LOGBUFFER wait types.

Sometimes, we get this wait type during database restore operation as well. As we know backup operation captures details like database file sizes irrespective of how much data or logs filled inside the data or log file. When you execute restore operation, it will create that much big file and it will take more time because SQL Server will be busy in zeroing out the file equivalent to source data or log file size. If you have enabled IFI on your database server then you will not be getting this wait type for data files but still you might get it for log file size allocation.

Suppose your database has 100GB of log file and only 12 GB is filled. If you take backup of this database and restore it anywhere then restore operation will create a log file of 100GB that will take lot much time because of zero initialized operation to be performed for all 100GB size.

Solutions to Reduce Wait Type PREEMPTIVE_OS_WRITEFILEGATHER

Microsoft has given a feature Instant File Initialization to overcome or reduce this wait type but this feature works for only data files not log files. We hope this feature will work on both data as well as log files in future releases of SQL Servers.

If you are getting this wait type during extending data files or during data file autogrowth then you can simply enable Instant File Initialization on your SQL Server database server and this wait type will disappear from next time. This feature will skip zero-initialization of data files and directly allocate given size to the data files. Read attached article to learn Instant File Initialization and how to enable it on your SQL Server Instance?

But we know this feature does not work on log files so we cannot use this feature to reduce this wait type if it occurs during extending log file or during log file autogrowth. We faced same issue because we had already enabled Instant File Initialization for data files but stuck during extending log files. Here I will give you few best practices that you can apply to avoid and reduce this wait type.

1. Increase auto-growth from the default value to some numeric value to reduce the frequent auto-growth occurrence for your database file. Thumb rule is to keep autogrowth size one eight of the corresponding database-file size. Read this article to know the Perfect value for database file Autogrowth?
2. If possible avoid auto-growth occurrence by allocating expected size grown in future to the database.
3. Monitor Storage subsystems and Investigate if any issues identified. Upgrade Storage if required.
4. Have a look at the Windows Event Viewer for any system related issues.
5. Enable Instant File Initialization to avoid zero-initialization for data files.
6. Consolidate transactions from row-by-row statements to batch processing.
7. If you have bigger log file having small number of logs filled inside. You can check log file size using DBCC SQLPERF command given in attached article. You can shrink the log file and take the backup again to perform the restore. This should be a quick process rather than waiting for zeroing to complete for previous size of log file.
8. Stop any activity to free-up the target disk’s header so that it can dedicatedly work on zeroing out the LDF file.

Read More:

• How to Speed up Huge Data Load in SQL Server?
• Understanding ASYNC_IO_COMPLETION Wait Type
• Manage Transaction Log File during Data Load
• Understanding SQL Server Wait Types
• How to reduce RESOURCE_SEMAPHORE wait type?

I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post How to Reduce PREEMPTIVE_OS_WRITEFILEGATHER Wait Type in SQL Server? appeared first on Techyaz.com.

1. Get log space size
2. View wait statistics data
3. Clear wait statistics data
4. Clear latch statistics data

Let’s discuss each of above tasks as use cases of DBCC SQLPERF command.

Read More

• How to get total no of VLFs (Virtual Log Files) inside Log file?
• Understanding Data Files, Pages & Extents
• How to Avoid Page Split in SQL Server?

Get Transaction Log space using DBCC SQLPERF

We can get transaction log space utilization details like how large your transaction logs are and what percentage of them is in use by executing dbcc sqlperf command. Run below command to get the log space details of all database hosted your SQL Server Instance.

--Get Transaction Log Size Details
DBCC SQLPERF(logspace)

You can see the below screenshot; all the columns are self-explanatory. We can see percent of log space used out of total space allocated to the log file.

View and Clear Wait Statistics Data

We can get wait statistics data using dbcc sqlperf command as well. We used to get these details using dbcc sqlperf command in older version of SQL Server instances. Although, you can still get wait statistics data using this command in current versions of SQL Server but now there is a better way to get such details using DMV sys.dm_os_wait_stats.  Here, I will show you wait stats data using DMV only. Run below commands to get wait statistics details of your SQL Server instance.

--Get wait Statistics using DBCC command.
DBCC SQLPERF(waitstats)

--Get Wait Statistics using DMV sys.dm_os_wait_stats
SELECT * FROM sys.dm_os_wait_stats
Order by wait_time_ms desc

You can see the details about all wait types on your SQL Server instance. You can also see the total wait time in milliseconds. These information is very important to deal with performance issues but details like total wait time, total waiting tasks count are not the real-time values. These values are accumulated since last SQL Server service start or last time wait stats was reset using dbcc sqlperf command. It is highly recommended to not clear wait stats data until it is required to do that otherwise you will lose valuable information about wait statistics. Read attached article about why should you not clear wait statistics data stored in DMV sys.dm_os_wait_stats.

If you want to get the correct waiting task counts or waiting time then you need to clear the wait statistics data using dbcc sqlperf command. Here, I will show you the difference after clearing the wait stats. Run below command to clear all wait stats data on your SQL Server Instance.

DBCC SQLPERF(“sys.dm_os_wait_stats”, CLEAR)

You can see I have successfully ran this command in below screenshot.

Now we will again run same command to get the wait statistics data. I ran it and I get below data that is showing different than the one showing before clearing wait stats. This way you can better analyze your database performance and waiting tasks and time to reach on any conclusion.

Clear Latch Statistics Data

I described about clearing wait stats data in above section. Here, I will show you how to clear latch statistics data. Latches are locks for RAM memory structures to prevent concurrent access and ensure serial execution of kernel code. You can run SQL command to check current latch statistics data.

--Get latch statistics data
SELECT * FROM sys.dm_os_latch_stats
ORDER BY wait_time_ms desc

You can see current latch statistics details that is also not showing only real time value but it is accumulated data since last SQL Server service restarted or last time latch wait statistics was reset/cleared.

Now we will clear these latch statistics data to get real time value of latch details. Run below dbcc sqlperf command to clear the latch statistics.

DBCC SQLPERF(“sys.dm_os_latch_stats”, CLEAR)

You can see we have successfully executed above command on our SQL Server Instance.

Now go ahead and run same SQL command to get the latch statistics again. You can see now there is no waiting requests on our SQL Server Instance and you are getting real time value about latches.

Related Articles

• Should We Clear Wait Stats Data from DMV sys.dm_os_wait_stats?
• How to Fix BACKUPBUFFER Wait Type?
• How to Improve Bulk Data Load Performance?

These are the 4 use cases of DBCC SQLPERF command that we use during different database activities. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post 4 Different Usages of DBCC SQLPERF in SQL Server appeared first on Techyaz.com.

There is new feature called Resumable Online Index Rebuild introduced in SQL Server 2017 that is an extension of online index rebuild operation. Learn about Resumable Online Index Rebuild operation in SQL Server 2017 in attached article.

Read More:

• What should be best value for fill factor in SQL Server?
• How to Avoid Page Split in SQL Server?
• How to use Resumable Online Rebuild Operation?

ONLINE Index Rebuild Operation

Rebuilding an index drops and re-creates the index. Indexes becomes unavailable during this exercise that causes an outage for that object. Microsoft has given a solution to avoid inaccessibility of the object during index rebuild operation. We can specify ONLINE keyword value as ON in CREATE INDEX or ALTER INDEX to rebuild indexes online.

When you are rebuilding an index and the ONLINE option is set to ON, the underlying objects, the tables and associated indexes, are available for queries and data modification. Row versioning is used to allow for transactional consistency. SQL Server reindex ONLINE option does not work if your table has BLOB data types or index is an XML index or Spatial index.

I have given the list of T-SQL statements that will rebuild indexes ONLINE on your table. You just need to change the index name, table name and column name from below statements.

Below SQL code will rebuild the clustered index on column EMPID on the table dbo.Salary. As we are rebuilding this cluster index using dropping and recreating the cluster indexes so existing index will be dropped in the process, but it will be available during the operation because the ONLINE option is specified.

CREATE CLUSTERED INDEX CI_Salary_EMPID ON dbo.Salary(EMPID)
WITH(DROP_EXISTING = ON, ONLINE = ON)

We can also use ALTER INDEX statement to rebuild a specific index or all indexes on a specified table. The ONLINE syntax remains the same as the CREATE INDEX statement if you want to run this operation online.

ALTER INDEX ALL ON [dbo.Salary]
REBUILD WITH(ONLINE = ON)

Above statement will rebuild all indexes on table Salary keeping everything accessible during this operation. Now, we can also rebuild clustered index only on the dbo.Salary table. The ONLINE option is specified, which means that the table will be accessible during the rebuild operation.

ALTER INDEX CI_Salary_EMPID ON dbo.salary
REBUILD WITH(ONLINE = ON)

But if we will remove ONLINE option from above statement, then table will NOT be accessible during rebuild operation. You can run below command to validate the accessibility of your table for which you are rebuilding the indexes.

ALTER INDEX CI_Salary_EMPID ON dbo.salary
REBUILD

Disk Space Considerations 

Disk space is an important consideration when you create, rebuild, or drop indexes. Inadequate disk space can degrade performance or even cause the index operation to fail. There might be other major performance impact of Online Index Rebuild operation as well. Please consider below recommendations if you are using Online index operations or during SQL Server rebuild index online.

1. During ONLINE index rebuild and index creation, additional space is required in the data file to keep the second copy of the index. This is because SQL Server creates a snapshot of the index and once index rebuild or creation completes, second copy of this index will be removed by SQL Server.
2. Additional disk space is required for the temporary mapping index. This temporary index is used in online index operations that create, rebuild, or drop a clustered index.
3. Make sure the transaction log has been backed up and truncated before running index operations online, and that the log has sufficient space to store the projected index and user transactions.
4. Specify SORT_IN_TEMPDB option to ON for the index operation. This separates the index transactions from the concurrent user transactions. The index transactions will be stored in the tempdb transaction log, and the concurrent user transactions will be stored in the transaction log of the user database. Make sure to keep tempdb log and user database log files on separate drives to avoid any disk space issues.
5. Verify that the tempdb database and transaction log have sufficient disk space to handle the index operation. The tempdb transaction log cannot be truncated until the index operation is completed.
6. Do not run the online index operation in an explicit transaction. The log will not be truncated until the explicit transaction ends.

Performance Considerations

If we think about performance impact of Online Index Rebuild operation on SQL Server then it will put more burden on your database server than offline index operations. Because both the source and target structures are maintained during the online index operation.

ONLINE Index operations will be slower than equivalent offline index operations because insert, update, and delete transactions is increased, potentially up to double. This could cause a decrease in performance and greater resource usage, especially CPU time, during the index operation. Online index operations are enterprise features and are fully logged operations.

It is recommended to use legacy way of index rebuild or index operations if your system allows a maintenance window. Remember users will be restricted to access data during the offline index operation, but the operation finishes faster and uses fewer resources.

If you have a system that runs 24×7 then ONLINE index rebuild or creation will be a better choice but make sure to consider all points described in this article before choosing online index operations. You need to think from disk space and performance both prospect and design your system that can handle the load and sustain during peak hours.

Related Articles

• SQL Server Interview Questions & Answers on Indexes
• Understanding Different types of Indexes in SQL Server

Here, I have described Disk and Performance Impact of Online Index Rebuild operation on SQL Server. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post Disk and Performance Impact of Online Index Rebuild Operation appeared first on Techyaz.com.

Should We Clear Wait Stats Data in SQL Server?

We do not recommend to clear the wait stats data frequently because all valuable information about wait statistics will be vanished if you clear them. Always try to capture wait statistics information before clearing wait stats. This way you would have both before and after data. You can use sys.dm_os_wait_stats DMV to get the SQL Server wait statistics information.

Generally, we reset or clear wait stats information to capture actual data during a given time duration because this DMV does not show only current data. The DMV sys.dm_os_wait_stats accumulates all the counts and wait times since last time SQL Server started or its stats got cleared by dbcc sqlperf command.

Here I will show you how to clear the SQL Server wait statistics and the difference between data captured before clearing the wait stats and after clearing the wait stats.

Get Wait Statistics Data

The DMV sys.dm_os_wait_stats provides essential metrics for diagnosing SQL Server performance problems. This DMV captures all the details like waiting requests counts, signal waits and waiting time since last time SQL Server instance got started or last time you have cleared your wait stats using DBCC SQLPERF command. Run below command to get performance wait stats details of your SQL Server instance.

--Get Wait Statistics using DMV sys.dm_os_wait_stats
SELECT * FROM sys.dm_os_wait_stats
Order by wait_time_ms desc

You can see the details about all the wait types on your SQL Server instance. You can also see the total wait time in milliseconds. This data is very important if you are dealing with any performance issue but as i discussed above details like total wait time, total waiting tasks counts are the accumulated/incremental values.

Reset or CLEAR Wait Statistics

Now you have captured wait stats information since your SQL Server was started. If you want to get only current waiting task counts or waiting time then you need to reset or clear wait statistics data using dbcc sqlperf command. Run below command to clear all wait stats data on your SQL Server Instance.

DBCC SQLPERF(“sys.dm_os_wait_stats”, CLEAR)

You can see I have successfully executed this command in below screenshot.

Now we will again run same command to get the wait statistics data. I ran it and I get below information that is showing different than the one showing before clearing wait stats.

You can see how you will lose valuable information if you will clear your wait stats without capture its earlier data. This way you can also better analyze your database wait types, waiting tasks to reach on any conclusion to fix the issue.

Related Articles

• Understanding SQL Server Wait types
• Identify Disk Bottleneck in SQL Server using Perfmon Disk Counters
• Understanding Resource_Semaphore wait type

Now you understand that we should not reset or clear wait stats frequently because wait stats data is very crucial to deal with any performance issue. You can reset or clear the wait stats but make sure to capture the data before clearing the wait stats. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post Should We Clear Wait Statistics Data from DMV sys.dm_os_wait_stats in SQL Server? appeared first on Techyaz.com.

Related Articles:

• SQL Server Interview Q&A on Alwayson Availability Group
• SQL Server DBA Interview Questions & Answers
• SQL Server Backup & Recovery Interview Questions & Answers
• SQL Server Architecture Interview Questions & Answers
• Index Rebuild vs Index Reorganize Operation

SQL Server Interview Questions & Answers on Indexes

Question – How data stores in Indexes?

Answer – Data stores in indexes or in tables on a series of 8 kb data pages. Indexes Data on these data pages are organized in a B-Tree structure that supports fast retrieval of the rows, based on their index key values. Each page in an index B-tree is called an index node. The top node of the B-tree is called the root node. The bottom level of nodes in the index is called the leaf nodes. The leaf nodes of a nonclustered index is made up of index pages instead of data pages whereas in a clustered index, the leaf nodes contain the data pages.

Question – What is Fill Factor and what should be the perfect value for it?

Answer – A fill factor value determines the percentage of space on each leaf-level page to be filled with data, reserving the remainder on each page as free space for future growth. Fill factor values 0 and 100 are the same in all respects. The fill-factor option is provided for fine-tuning index data storage and performance. Although, there is no defined value that we can say is perfect value for fill factor. You can set it to somewhere around 80% and monitor fragmentation over time. Then, you can tweak its value up or down depending on how fragmented the indexes get. Read more about fill factor and its best value.

Question – Explain Page Split and whether it is good for SQL Server or bad?

Answer – Whenever you update existing rows with the data that is bigger in size to save on their data page then Page Split operation occur to make space for this new update. Page split reduces performance so we can say page split is not at all a good thing for our database.

Question – How page split is bad for performance?

Answer – Page split is a resource intensive operation and causes fragmentation that leads to deficient performance in terms of increased I/O operations. We should take it seriously if there are frequent page splits occur and consider reducing its occurrences

Question – What is Difference between clustered and nonclustered Indexes?

Answer – Clustered indexes sort and store the data rows in the table or view based on their key values. There can be only one clustered index per table, because the data rows themselves can be sorted in only one order.

Nonclustered indexes have the same B-tree structure as clustered indexes. The data rows of the underlying table are not sorted and stored in order based on their non clustered keys. The leaf layer of a nonclustered index is made up of index pages instead of data pages. You can create up to 999 nonclustered indexes on a table. 

Question – Does indexes always improve database performance?

Answer – NO, sometimes indexes can reduce your database performance if you are running a bulk data upload or your application has a nature of frequent data Insert operations. Indexes are very useful if you are accessing or reading the data.

Question – What is filter index?

Answer –  An optimized nonclustered index, especially suited to cover queries that select from a well-defined subset of data. It uses a filter predicate to index a portion of rows in the table. A well-designed filtered index can improve query performance, reduce index maintenance costs, and reduce index storage costs compared with full-table indexes.  

Question – What is covering index?

Answer – A covering index is one which can satisfy all requested columns in a query without performing a further lookup into the clustered index.

Question – Explain column store index?

Answer – An in-memory columnstore index stores and manages data by using column-based data storage and column-based query processing.

Columnstore indexes work well for data warehousing workloads that primarily perform bulk loads and read-only queries. 

Question – Why can’t a table have more than one clustered indexes?

Answer – Clustered indexes sort and store the data rows in the table or view based on their key values so you can’t store data in table based on two index key that’s why you can create only one cluster index on a table.

Question -What is Heap and when it is useful from clustered table?

Answer – Heap is a table without having cluster indexes. Heaps are generally useful if you are running huge bulk data load or frequent DML operations.

Question – How do you find about missing indexes that you need to create to improve database performance?

Answer – We can run SQL Server database Tuning Advisor that will suggests us about all the missing indexes that we should create to improve db performance. We can also use DMVs related to missing indexes sys.dm_db_missing_index_details, sys.dm_db_missing_index_groups and sys.dm_db_missing_index_columns to get these details.

I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post SQL Server Interview Questions & Answers on Indexes appeared first on Techyaz.com.

1. Excellent – You Disk Subsystem is responding perfectly for respective counter.
2. Good – You Disk Subsystem is responding fairly for respective counter and you can consider it good considering your workload and other resource utilization.
3. Poor – You Disk Subsystem is not responding well for respective counter and it requires your intervention to drill down more to understand the reason behind slowness.

If you want to read about CPU and Memory related performance counters then i would suggest you to go through with below attached articles.

• Top Perfmon Counters to Identify SQL Server CPU Bottleneck
• Top 10 Perfmon Counters to Identify SQL Server Memory Pressure

Below are the list of important perfmon disk counters that we will be discussing in this article.

• Average Disk Sec/Read
• Average Disk Sec/Write
• Average Disk Sec/Transfer
• Average Disk Queue Length
• Current Disk Queue Length
• Disk Read/Sec and Disk Write/Sec
• %Disk Time
• %Idle Time

Average Disk Sec/Read

Average Disk Sec/Read is important perfmon disk counter that shows the average time in seconds needed to read data from disk and better explain the disk latency. We can find these counters at the logical and physical disk object level. The value for this counter is generally the number of seconds it takes to do each read. The shorter the time needed to read or write data, the faster the system. If you constantly get this counter value more than 20ms then it means your disk is not performing well.

Your disk performance is:

• Excellent, if the value of this counter is less than 6 milliseconds (0.006 sec)
• Good, if the value of this counter is up to 15 milliseconds (0.015 sec)
• Poor, if the value is constantly more than 15 milliseconds (0.015 sec)

Average Disk Sec/Write

Average Disk Sec/Write is also an important counter that shows the average time in seconds needed to write data to disk. that can better explain the disk latency. This counter can be tracked at the logical and physical disk level. The value for this counter is generally the number of seconds it takes to do each read. The shorter the time needed to read or write data, the faster the system. If you constantly get this counter value more than 20ms then it means your disk is not performing well.

Your disk performance is:

• Excellent, if the value of this counter is less than 6 milliseconds
• Good, if the value of this counter is up to 15 milliseconds
• Poor, if the value is constantly more than 15 milliseconds.

Average Disk Sec/Transfer

Average Disk Sec/Transfer is a time that measures average latency for read or write operations. If the Average Disk Sec/Transfer value increases, this indicates that the I/O subsystem is not optimally keeping up with the I/O demand. This counter is available under Logical Disk and Physical Disk object. If you are keeping a watch on this counter make sure to have a look at above perfmon disk counters Average Disk Sec/ Read and Average Disk Sec/Write as well because Average Disk Sec/Transfer is the average of both read and write counters so you need to find out high value is due to read operation or write operation. Below are the values that will tell you whether your disk is performing excellent, good or it is performing slow and requires your intervention to improve its performance.

Your disk performance is:

• Excellent, if the value of this counter is less than 6 milliseconds
• Good, if the value of this counter is up to 15 milliseconds
• Poor, if the value is constantly more than 15 milliseconds.

Average Disk Queue Length

Average Disk Queue Length counter shows you the average number of read and write requests that were queued on the selected physical disk. If this value is exceeding its threshold frequently then we should also look at other perfmon disk counters Average Disk Read Queue Length and Average Disk Write Queue Length. This way you can diagnose whether Average Disk Queue Length is high due to high read or high write operation. The recommended value for this counter is less than 2 per individual disk so if you have multiple disks then your recommended value for Average disk queue length would be twice the no of disks.

Be careful when using thresholds of 2 with SQL Server and SANs.  This value can be much higher than 2 if you have multiple disks.  Let’s say you have 6 drives and Average disk queue length counter is showing around 12, then you should consider your Average disk queue length value 9/6 = 1.5 not 9.

Your disk performance is:

• Excellent, if the value of this counter is less than 1.5 per drive.
• Good, if the value of this counter is up to 2 per drive.
• Poor, if the value is constantly more than 2 per drive,

Current Disk Queue Length

Current Disk Queue Length is the number of disk requests that are currently waiting as well as requests currently being serviced. The difference between average disk queue length and current queue length is CDQL tells current load whereas ADQL tells load during a given time frame. The Current Disk Queue Length metric in Windows Performance Monitor is available for both physical and logical disk. The Current Disk Queue Length value should be less than 2 per disk.

Your disk performance is:

• Excellent, if the value of this counter is less than 1.5 per drive.
• Good, if the value of this counter is up to 2 per drive.
• Poor, if the value is constantly more than 2 per drive.

Disk Read/Sec and Disk Write/Sec

Disk Reads/Sec and Disk Writes/Sec show you the number of read and write operations happening per second on the disk. If you want to set the benchmark for disk capacity then you can determine it by gradually increasing the load on the system.

Your disk performance is:

• Excellent, if the value of this counter is less than 75% of total disk capacity.
• Good, if the value of this counter is up to 85% of total disk capacity.
• Poor, if the value is constantly more than 85% of total disk capacity.

%Disk Time

%Disk Time counter tells how busy the disk is in performing read and write operations. You can see its values normal even your disk has serious disk performance issue. You should compare its value with current/average disk queue length before reaching on any conclusion about disk issue.

Your disk performance is:

• Excellent, if the value of this counter is less than 75% of total disk capacity.
• Good, if the value of this counter is up to 85% of total disk capacity.
• Poor, if the value is constantly more than 85% of total disk capacity.

%Idle Time

%Idle Time measures the percentage of time the disk was idle during the sample interval. The disk is idle when it’s not processing read and write requests. If you are constantly getting its value less than 40% then you can either move some applications from your machine to another machine if you are running other applications as well apart from SQL Server or You can change the current disk system with a faster disk system.

Your disk performance is:

• Excellent, if the value of this counter is greater than 60% of total disk capacity.
• Good, if the value of this counter is greater than 50% of total disk capacity.
• Poor, if the value is constantly less than 40% of total disk capacity.

Related Articles:

• Disk and Performance Impact of running Online Index Rebuild Operation
• How to fix IO error: SQL Server has encountered N occurrence(s) of I/O requests taking longer than 15 seconds to complete.
• Improving Bulk Data Load Performance in SQL Server
• Understanding ASYNC_IO_COMPLETION Wait Type

I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post Identify Disk Bottleneck in SQL Server using Perfmon Disk Counters appeared first on Techyaz.com.

SQL Server has encountered 32 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [E:\MSSQL\Data\database_file.mdf] in database [Database_Name] (%d). The OS file handle is 0x000000F543. The offset of the latest long I/O is: 0x000000004h583.

This message indicates that SQL Server has issued a read or write request from disk, and the I/O requests taking longer than 15 seconds to return. Microsoft SQL Server reports Event Id 833 to the event log, and above text in the ERRORLOG file along with other symptoms like high wait times for PAGEIOLATCH waits or other disk IO related wait types in SQL Server.

Why we get this issue/warning?

SQL Server database is designed to perform intensive disk I/O operations. An I/O operation may be either a read or a write operation and the standard disk speed to perform these operations is around 10 milliseconds or less. If any of the read or write operation takes more time to perform their activity and generally disk speed reaches or exceeds 15 seconds of time, then database engine reports event id 833 along with above log information in error log file.

If you receive such messages, it does not mean problems with SQL Server. They are reported to let DBAs and system administrator know about poor storage performance and help them to investigate why the I/O request took so long. Long I/Os often indicate a SQL Server workload that is too intense for the disk subsystem. Disk Subsystems perform poorly/slowly because of either stuck or stalled IO operations. Before going ahead let’s first understand about stuck and stalled IO operation.

Stuck I/O

Stuck I/O is defined as an I/O request that does not finish.  To resolve a stuck I/O condition, you must usually restart the computer or perform a similar action. A stuck I/O condition typically indicates one of the following:

• Faulty hardware
• A bug in an I/O path component

Stalled I/O

Stalled I/O is defined as an I/O request that does finish or that takes excessive time to finish. Stalled I/O behavior typically occurs because of one of the following reasons:

• The hardware configuration
• The firmware settings

Below is the list of items that can cause stuck or stalled I/O operations on disk subsystem that can adversely affect SQL Server performance.

• Faulty hardware
• Incorrectly configured hardware
• Firmware settings
• Filter drivers
• Compression
• Bugs
• All transactions are in one drive like (backup, live database updates)
• Blocking
• Latch contentions
• Missing Indexes in databases

Resolution

Disk I/O problems are one of the most difficult problems to diagnose and to find the root cause. There might be multiple reasons due to which your disk subsystems perform slow. First step towards fixing such issue is to examining the event logs for any hardware related error messages and then proceed towards fix accordingly. Follow below steps to reduce the IO overheads on your database server if I/O requests taking longer than 15 seconds or perfoming slow.

1. Check & update all missing device drivers and firmware with the latest updates.

2. Analyze memory pressure and CPU usages trends as well because if you will run with CPU or memory pressure, there might be possibility slow disk performance.

3. Monitor disk I/O performance counters either using Performance Monitor or using DMVs to examine your disk subsystem workload. Read attached article to get the optimum values for all given counters. It’s highly recommended to monitor these counters for a while, and then compare it to your disk baselines value that you had captured during stress testing or load testing exercise.

• Average Disk Sec/Transfer
• Average Disk Sec/Read
• Average Disk Sec/Write
• Average Disk Queue Length
• Current Disk Queue Length
• Disk Read/Sec
• Disk Write/Sec
• %Disk Time

4. Microsoft suggests being careful when you use the performance counters because SQL Server takes full advantage of asynchronous I/O capabilities that push the disk queue lengths heavily. Therefore, longer disk queue lengths alone do not indicate a problem. So, compare every counter with their baseline and then analyze the trend.

5. Monitor SQL Server Wait Types, If the disk is slowing all the times then you will get wait types PAGEIOLATCH, WRITELOG or LOGBUFFER with high wait time.

6. Exclude SQL Server Data and Log files from antivirus software scans. If your database files are configured to be scanned by antivirus software then this will have slowed your disk performance.

7. If you are getting this issue on a specific time then I would suggest checking the conflict between your SQL Server Agent Jobs? There might be some data load/Disk IO intensive query or database maintenance activities will be running parallelly at the same time. Change the Agent jobs schedule time carefully so that they should not conflict to each other.

8. Don’t place database files and backups on single drive. Always place data file, log files and tempdb files on separate drives to reduce disk IO overhead. Also, it is highly recommended to run backups on separate drive because if you run backups on data drives then this can reduce disk performance.

9. Check your database files property for which you are getting this error. The size of database files should not be limited to fixed size with auto growth disabled. Enabling the auto growth of all the data files with appropriate value can fix this issue if you have put limits of database files growth. Read the attached article to understand the best value for database file autogrowth.

10. Disk I/O can perform poorly if there is no proper indexing on your tables. You can use Index Turning Wizard to resolve I/O pressure on the system. You can find missing indexes from most used or accessed tables and create appropriate indexes to reduce query time and resources overhead.

Disable this Error to Log using Trace flag 830

Microsoft has given us an option to disable stalled or stuck I/O detection by using trace flag 830, although we do not recommend that you do this.

To disable detection when SQL Server starts, use the -T830 startup parameter to disable detection every time that SQL Server is started. To disable detection for an instance of SQL Server that is currently running, use the following statement:

dbcc traceoff(830, -1)

This setting is effective only for the life of the SQL Server process.

Related Articles:

• Identify Disk Bottleneck in SQL Server using Perfmon Disk Counters
• How to Improve Bulk Data Load Performance?
• What is ASYNC_IO_COMPLETION wait type?
• Understanding BACKUPBUFFER wait type

I hope you like this article. Please follow our Facebook and Twitter handle to get latest updates.

The post Fix Event ID 833: SQL Server has encountered 32 occurrence(s) of I/O requests taking longer than 15 seconds to complete. appeared first on Techyaz.com.

What is Page Split?

Whenever you update existing rows with the data that is bigger in size to save on their data page then Page Split operation occur to make space for this new update. Data from existing data page first move to new page that is added during Page Split operation and make room to accommodate new records. That is why this operation is resource intensive and create fragmentation. Page splits only occur when data changes in the database. If you are adding all your data at the end of the table then page split will not occur.

If your fill factor is 100 or 0, it will work in a same way. That means there is no room in leaf level page to accommodate future updates. Setting fill factor value to 0 or 100 will increase page split occurrences.

How to Check Page Split in SQL Server?

Here, i will explain how to check page split in SQL Server. Whenever Page split occurs, it will be logged in transaction log file as LOP_DELETE_SPLIT operation. We use fn_dblog function to read SQL Server transaction log file. Run below T-SQL code to see all page splits occurrence using transaction log file.

SELECT *
FROM
fn_dblog (NULL, NULL)
WHERE
[Operation] = N'LOP_DELETE_SPLIT'

You can see LOP_DELETE_SPLIT is logged under operations column. That means it’s a page split operation.

Paul Randel has given very good sql code to check total number of page splits occurrence. Below is the code you need to run to get this details.

SELECT
[AllocUnitName] AS N'Index',
(CASE [Context]
WHEN N'LCX_INDEX_LEAF' THEN N'Nonclustered'
WHEN N'LCX_CLUSTERED' THEN N'Clustered'
ELSE N'Non-Leaf'
END) AS [SplitType],
COUNT (1) AS [SplitCount]
FROM
fn_dblog (NULL, NULL)
WHERE
[Operation] = N'LOP_DELETE_SPLIT'
GROUP BY [AllocUnitName], [Context];
GO

You can check total number of page splits are shown for each indexes in below screenshot. Now we will discuss how to avoid page split in SQL Server for such indexes.

How to Reduce Page Split Occurrences?

An index that has many random inserts and has very full pages will have an increased number of page splits. This causes more fragmentation and that is very bad for performance. We can reduce page splits by keeping some room in the data pages by setting fill factor value less than 100. The extra bytes on each page will help to minimize page splits caused by extra length in the rows. Read attached article to understand the best value for fill factor to avoid such page-split operations.

We can also use index rebuild operation with the FILL FACTOR option that will allow the page fullness to be changed to fit the query pattern on the index. This will also helpful in reducing total page-split counts.
I hope you like this article. You can drop your questions in comment section. Please follow our Facebook page and Twitter handle to get latest updates.

The post How to Avoid Page Split in SQL Server? appeared first on Techyaz.com.

Index Rebuild vs Index Reorganize

Index Rebuild operation first drops and then recreates the index. This removes fragmentation, reclaims disk space by compacting the pages based on the specified or existing fill factor setting. Microsoft recommends Index Rebuild operation to defrag indexes if the fragmentation level of your index is greater than 30%. You can execute Index rebuild operation online as well as offline. Online index rebuild operation is enterprise only feature and you can use it if you have SQL Server Enterprise edition.

Reorganizing an index uses minimal system resources. It defragments the leaf level of clustered and nonclustered indexes on tables and views by physically reordering the leaf-level pages to match the logical, left to right, order of the leaf nodes. Microsoft recommends using Index Reorganize operation to defrag indexes if the fragmentation level of your index is greater than 5% and less than or equal to 30%. Reorganizing an index is always executed online.

Read More:

• What should be best value for fill factor in SQL Server?
• How to Avoid Page Split in SQL Server?
• SQL Server Interview Questions & Answers on Indexes

How to Check Fragmentation Level?

First, we detect the fragmentation level of the indexes. Once we have the details about fragmented indexes then we can decide whether to rebuild that index or reorganize it. We can get index fragmentation level using DMV sys.dm_db_index_physical_stats or by launching dashboard report. Find the average fragmentation percentage of all indexes in the table dbo.Salary of database techyaz by running below T-SQL script.

USE Techyaz;
GO
SELECT a.index_id, name, avg_fragmentation_in_percent
FROM sys.dm_db_index_physical_stats (DB_ID(N'Techyaz'),
OBJECT_ID(N'dbo.salary'), NULL, NULL, NULL) AS a
JOIN sys.indexes AS b
ON a.object_id = b.object_id AND a.index_id = b.index_id;
GO

We can see the output of this code in below image. As i have only one table and two indexes in my database but if you have multiple tables and indexes then output will look accordingly.

Another way to check the fragmentation level of your indexes is by running dashboard report.  Right click on your database, click on “Reports” then “Standard Reports” then finally click on “Index Physical Statistics” report. You can see this in below screenshot.

Once you click on “Index Physical Statistics” report, a dashboard report will run and appear on right side pane as per below screenshot. You will get fragmentation level for each index here.

How to Remove Fragmentation Level?

There are two ways to remove fragmentation level of indexes. These are by either using Index Rebuild Operation or by using Index Reorganize Operation. Here, i will show you how to remove fragmentation using both ways one after another.

Remove Fragmentation using Index Reorganize Operation

We can see our index has very low fragmentation level that is less than 30% in above images so we will reorganize it rather than rebuild. We will rebuild only those indexes that have more than 30% of fragmentation level. Run below T-SQL command to reorganize index IX_Salary_EMPID.

USE Techyaz;
GO

ALTER INDEX IX_Salary_EMPID
ON dbo.Salary
REORGANIZE ;
GO

If you want to reorganize all indexes of this table dbo.salary in one shot, you can use ALL keyword in Alter Index command. Run below command to reorganize all indexes of table dbo.salary in one go.

USE Techyaz;
GO

ALTER INDEX ALL ON dbo.Salary
REORGANIZE;
GO

Remove fragmentation using Index Rebuild Operation

Similarly, we will rebuild all those indexes that are highly fragmented. As I don’t have any such index but if you will get any index that has more than 30% of fragmentation then you can run below command to rebuild that index.

USE Techyaz;
GO
ALTER INDEX PK_Salary_EMPCODE ON dbo.Salary
REBUILD;
GO

If you want to REBUILD all indexes of dbo.salary table in go then again we will use ALL keyword in ALTER Index statement as we did in above example during REORGANIZE.

USE Techyaz;
GO
ALTER INDEX ALL ON dbo.Salary
REBUILD WITH (FILLFACTOR = 80, SORT_IN_TEMPDB = ON,
STATISTICS_NORECOMPUTE = ON);
GO

If your system does not allow any maintenance window and run 24×7 then you can use online method to rebuild these indexes. Read attached article to learn how to rebuild indexes online in SQL Server. Even SQL Server 2017 has given more control on index rebuild operation by introducing a new feature called Resumable Online Index Rebuild. Read the attached article to learn about this feature in which we can pause a index rebuild operation and resume it later from the point it was stopped.

I hope you like this article. You can drop your questions and feedback in comment section. Please follow our Facebook page and Twitter handle to get latest updates.

The post Difference between Index Rebuild and Reorganize appeared first on Techyaz.com.

• Root Level
• Intermediate Level
• leaf Level.

Clustered index leaf-level pages contain the data in the table whereas Non Clustered index leaf-level pages contain the key value and a pointer to the data row in the clustered index or heap.

Read More

• How to use Resumable Online Index Rebuild Operation?
• Understanding Online Index Rebuild Operation
• SQL Server Interview Questions & Answers on Indexes

If your table does not have SQL Server indexes and you want to access data from the table then database engine will take more time and performance because full table scan will be performed to get that data. However, if your table has an index on column, SQL Server can seek directly on the value and retrieve the rows.

Index Types

There are multiple types of indexes in SQL Server. Below is the list of such indexes.

Clustered Index

• Clustered indexes sort and store the data rows in the table or view based on their key values. There can be only one clustered index per table, because the data rows themselves can be sorted in only one order.
• The only time the data rows in a table are stored in sorted order is when the table contains a clustered index. When a table has a clustered index, the table is called a clustered table. If a table has no clustered index, its data rows are stored in an unordered structure called a heap.

Nonclustered Index

• Nonclustered indexes have the same B-tree structure as clustered indexes.
• The data rows of the underlying table are not sorted and stored in order based on their nonclustered keys.
• The leaf layer of a nonclustered index is made up of index pages instead of data pages.
• You can create upto 999 nonclustered indexes on a table.

Memory-optimized nonclustered indexes

• For memory-optimized nonclustered indexes, memory consumption is a function of the row count and the size of the index key columns

Hash Index

• With a hash index, data is accessed through an in-memory hash table. Hash indexes consume a fixed amount of memory, which is a function of the bucket count.

Unique  Index

• A unique index ensures that the index key contains no duplicate values and therefore every row in the table or view is in some way unique. Uniqueness can be a property of both clustered and nonclustered indexes.

Columnstore Index

• An in-memory columnstore index stores and manages data by using column-based data storage and column-based query processing.
• Columnstore indexes work well for data warehousing workloads that primarily perform bulk loads and read-only queries.
• Use the columnstore index to achieve up to 10x query performance gains over traditional row-oriented storage, and up to 7x data compression over the uncompressed data size.

Index with included columns

• A nonclustered index that is extended to include nonkey columns in addition to the key columns.

Index on computed columns  

• An index on a column that is derived from the value of one or more other columns, or certain deterministic inputs.

Filtered Index

• An optimized nonclustered index, especially suited to cover queries that select from a well-defined subset of data. It uses a filter predicate to index a portion of rows in the table. A well-designed filtered index can improve query performance, reduce index maintenance costs, and reduce index storage costs compared with full-table indexes.

Spatial  Index

• A spatial index provides the ability to perform certain operations more efficiently on spatial objects (spatial data) in a column of the geometry data type. The spatial index reduces the number of objects on which relatively costly spatial operations need to be applied.

XML Index

• A shredded, and persisted, representation of the XML binary large objects (BLOBs) in the xml data type column.

Full-text Index

• A special type of token-based functional index that is built and maintained by the Microsoft Full-Text Engine for SQL Server. It provides efficient support for sophisticated word searches in character string data.

How SQL Server Indexes are used by Query Optimizer

When a query is executed, the query optimizer evaluates each available method for retrieving the data and selects the most efficient method. The method may be a table scan, or may be scanning one or more indexes if they exist. When performing a table scan, the query optimizer reads all the rows in the table, and extracts the rows that meet the criteria of the query. A table scan generates many disk I/O operations and can be resource intensive. However, a table scan could be the most efficient method if, for example, the result set of the query is a high percentage of rows from the table.

When the query optimizer uses an index, it searches the index key columns, finds the storage location of the rows needed by the query and extracts the matching rows from that location. Generally, searching the index is much faster than searching the table because unlike a table, an index frequently contains very few columns per row and the rows are in sorted order.

Your task is to design and create SQL Server indexes that are best suited to your environment so that the query optimizer has a selection of efficient indexes from which to select. SQL Server provides the Database Engine Tuning Advisor to help with the analysis of your database environment and in the selection of appropriate indexes.

I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post Understanding SQL Server Indexes appeared first on Techyaz.com.