• Fix SQL Server Error 701: There is Insufficient System Memory to Run this Query

We can use Windows PerfMon utility to gather current values of all given performance counters. Just type perfmon in run prompt to launch perfmon tool to gather memory related performance counters. Although, Microsoft has also given a DMV related to performance counters that is very useful in collecting data and detecting CPU, IO or memory bottleneck in SQL Server. You can run below command to get all performance counters and their respective values using DMV sys.dm_os_performance_counters. You can also use this DMV to get performance counter values to identify memory pressure in SQL Server.

Select * from sys.dm_os_performance_counters

By default, SQL Server changes its memory requirements dynamically, on the basis of available system resources. I have explained SQL Server Memory Management and how SQL Server allocated memory to their transactions in below attached article.

• SQL Server Memory Allocation & Role of MIN/MAX Server Memory Configuration

Performance Counters to Identify SQL Server Memory Pressure

Let’s discuss top perfmon counters for memory usages to identify its bottleneck. Below is the list of top 10 performance counters to identify SQL Server Memory pressure. If you want to read CPU and Disk IO related perfmon counters then i would suggest you to read below attached articles.

• Top Perfmon Counters to Identify SQL Server CPU Bottleneck
• Identify Disk Bottleneck in SQL Server using Perfmon Disk Counters

Memory: Available Bytes

First thing you should look at in case of memory bottleneck in SQL Server is the available memory that can be used by processes. You can get this value by looking at the counter Available Bytes under object Memory. Low value of this counter shows the memory shortage on this server.  Now, you should focus behind the reason of this memory shortage. There might be possibility that some of your applications running on your server are not releasing memory or your system has not enough memory to perform current workload that leads to the memory bottleneck in SQL Server.

Memory: Pages/sec

This counter indicates the total number of pages that are either read from disk or written to the disk due to page fault. When a process requests a data page that is not in memory then it will be retrieved from disk to memory. This process is known is page fault. High value of this counter could indicate the excessive paging. We should further monitor Memory: Page Faults/sec counter to go indepth information. The average Pages/sec value should be below 50. A high value of Pages/sec indicates intensive memory activities like read from and write to disk. It means transactions are getting enough memory that is why they are using disk to process their requests.

Memory: Page Faults/sec

High value of counter Memory: Pages/sec could indicate the excessive paging. Next, we should monitor the Memory: Page Faults/sec counter to make sure that the disk activity is not caused by paging. If you want to determine whether SQL Server or another process is causing excessive paging then we should monitor the Process: Page Faults/sec counter for the SQL Server process instance. This will give an exact picture whether SQL Server is causing this paging issue or not.

Process: Working Set

This counter shows the amount of memory a process can use. This counter value shows whether you have set optimum value in min server memory and max server memory configuration or not. If this value shows constantly below min server memory and max server memory then you have configured too much memory for your SQL Server.

SQL Server: Buffer Manager: Buffer Cache Hit Ratio

This is very important counter and this counter shows the number of percentage that has been satisfied using data cache. Suppose the value of buffer cache hit ratio for your server is 94 it means 94% of total requests for data have been processed using data cache.

The ideal value for this counter is more than 90. Higher the value better the memory health.

SQL Server: Buffer Manager: Page Life Expectancy

Page Life Expectancy is very important counter to measure memory pressure for your system. THis counter indicates the average number of seconds a data page to remain in cache. Ideally, we have read multiple blogs to consider 300 or more as acceptable value but this value varies depending on your system configuration.

The ideal value is more than 300 but always compare this value with your system configuration. Higher the better concept is also applying here.

SQL Server: Buffer Manager: Free List Stalls/sec

This counter also shows a sign of memory pressure or insufficient memory on your server. This displays the number of requests per second where data requests stall because there are no buffers available. Any value greater than two indicates that SQL Server needs more memory.

SQL Server: Buffer Manager: Lazy Writes/sec

This counter tells about total number of Lazy writer performed in a second.  It means total number of times Lazy Writer process moves dirty pages from the buffer to disk. A lower number is better than a higher number. If value of this counter shows higher value continuously more than 20, then we need to add more memory to the system or we should think about optimizing memory intensive operations.

SQL Server: Buffer Manager: Checkpoint Pages/sec

This counter monitors the number of dirty pages per second, which are flushed to disk when SQL Server invokes the checkpoint process. High values for this counter may indicate memory pressure or insufficient memory.

SQL Server: Memory Manager: Memory Grants Pending

This counter shows the number of processes waiting for memory grant per second. Any value higher than zero indicates a lack of memory. We can also get this information by using a DMV sys.dm_exec_query_memory_grants. Those processes who are waiting to be granted the required memory will show you waiting with RESOURCE_SEMAPHORE wait type. I have explained about resource semaphore wait type in below attached article.

• Fix RESOURCE_SEMAPHORE Wait Type in SQL Server

Here, I have described top 10 performance counters to identify SQL Server memory pressure on your database server. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

Read More:

• Fix Error 833: SQL Server has encountered 32 occurrences of I/O requests taking longer than 15 seconds to complete.
• Disk and Performance Impact of running Online Index Rebuild Operation
• Understanding ASYNC_IO_COMPLETION Wait Type

The post Top 10 Performance Counters to Identify SQL Server Memory Pressure appeared first on Techyaz.com.

What is PREEMPTIVE_OS_WRITEFILEGATHER?

As per MSDN, PREEMPTIVE_OS_WRITEFILEGATHER is a wait type that occurs while SQL Server is waiting for the Operating System to complete portions of write operations. This wait is commonly seen in databases with auto-growth not configured appropriately or during large size allocation to database files. If you are getting this wait type frequently then you should investigate your SQL Server instance. You can also observe this wait type along with BACKUPBUFFER  and LOGBUFFER wait types.

Sometimes, we get this wait type during database restore operation as well. As we know backup operation captures details like database file sizes irrespective of how much data or logs filled inside the data or log file. When you execute restore operation, it will create that much big file and it will take more time because SQL Server will be busy in zeroing out the file equivalent to source data or log file size. If you have enabled IFI on your database server then you will not be getting this wait type for data files but still you might get it for log file size allocation.

Suppose your database has 100GB of log file and only 12 GB is filled. If you take backup of this database and restore it anywhere then restore operation will create a log file of 100GB that will take lot much time because of zero initialized operation to be performed for all 100GB size.

Solutions to Reduce Wait Type PREEMPTIVE_OS_WRITEFILEGATHER

Microsoft has given a feature Instant File Initialization to overcome or reduce this wait type but this feature works for only data files not log files. We hope this feature will work on both data as well as log files in future releases of SQL Servers.

If you are getting this wait type during extending data files or during data file autogrowth then you can simply enable Instant File Initialization on your SQL Server database server and this wait type will disappear from next time. This feature will skip zero-initialization of data files and directly allocate given size to the data files. Read attached article to learn Instant File Initialization and how to enable it on your SQL Server Instance?

But we know this feature does not work on log files so we cannot use this feature to reduce this wait type if it occurs during extending log file or during log file autogrowth. We faced same issue because we had already enabled Instant File Initialization for data files but stuck during extending log files. Here I will give you few best practices that you can apply to avoid and reduce this wait type.

1. Increase auto-growth from the default value to some numeric value to reduce the frequent auto-growth occurrence for your database file. Thumb rule is to keep autogrowth size one eight of the corresponding database-file size. Read this article to know the Perfect value for database file Autogrowth?
2. If possible avoid auto-growth occurrence by allocating expected size grown in future to the database.
3. Monitor Storage subsystems and Investigate if any issues identified. Upgrade Storage if required.
4. Have a look at the Windows Event Viewer for any system related issues.
5. Enable Instant File Initialization to avoid zero-initialization for data files.
6. Consolidate transactions from row-by-row statements to batch processing.
7. If you have bigger log file having small number of logs filled inside. You can check log file size using DBCC SQLPERF command given in attached article. You can shrink the log file and take the backup again to perform the restore. This should be a quick process rather than waiting for zeroing to complete for previous size of log file.
8. Stop any activity to free-up the target disk’s header so that it can dedicatedly work on zeroing out the LDF file.

Read More:

• How to Speed up Huge Data Load in SQL Server?
• Understanding ASYNC_IO_COMPLETION Wait Type
• Manage Transaction Log File during Data Load
• Understanding SQL Server Wait Types
• How to reduce RESOURCE_SEMAPHORE wait type?

I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post How to Reduce PREEMPTIVE_OS_WRITEFILEGATHER Wait Type in SQL Server? appeared first on Techyaz.com.

1. Get log space size
2. View wait statistics data
3. Clear wait statistics data
4. Clear latch statistics data

Let’s discuss each of above tasks as use cases of DBCC SQLPERF command.

Read More

• How to get total no of VLFs (Virtual Log Files) inside Log file?
• Understanding Data Files, Pages & Extents
• How to Avoid Page Split in SQL Server?

Get Transaction Log space using DBCC SQLPERF

We can get transaction log space utilization details like how large your transaction logs are and what percentage of them is in use by executing dbcc sqlperf command. Run below command to get the log space details of all database hosted your SQL Server Instance.

--Get Transaction Log Size Details
DBCC SQLPERF(logspace)

You can see the below screenshot; all the columns are self-explanatory. We can see percent of log space used out of total space allocated to the log file.

View and Clear Wait Statistics Data

We can get wait statistics data using dbcc sqlperf command as well. We used to get these details using dbcc sqlperf command in older version of SQL Server instances. Although, you can still get wait statistics data using this command in current versions of SQL Server but now there is a better way to get such details using DMV sys.dm_os_wait_stats.  Here, I will show you wait stats data using DMV only. Run below commands to get wait statistics details of your SQL Server instance.

--Get wait Statistics using DBCC command.
DBCC SQLPERF(waitstats)

--Get Wait Statistics using DMV sys.dm_os_wait_stats
SELECT * FROM sys.dm_os_wait_stats
Order by wait_time_ms desc

You can see the details about all wait types on your SQL Server instance. You can also see the total wait time in milliseconds. These information is very important to deal with performance issues but details like total wait time, total waiting tasks count are not the real-time values. These values are accumulated since last SQL Server service start or last time wait stats was reset using dbcc sqlperf command. It is highly recommended to not clear wait stats data until it is required to do that otherwise you will lose valuable information about wait statistics. Read attached article about why should you not clear wait statistics data stored in DMV sys.dm_os_wait_stats.

If you want to get the correct waiting task counts or waiting time then you need to clear the wait statistics data using dbcc sqlperf command. Here, I will show you the difference after clearing the wait stats. Run below command to clear all wait stats data on your SQL Server Instance.

DBCC SQLPERF(“sys.dm_os_wait_stats”, CLEAR)

You can see I have successfully ran this command in below screenshot.

Now we will again run same command to get the wait statistics data. I ran it and I get below data that is showing different than the one showing before clearing wait stats. This way you can better analyze your database performance and waiting tasks and time to reach on any conclusion.

Clear Latch Statistics Data

I described about clearing wait stats data in above section. Here, I will show you how to clear latch statistics data. Latches are locks for RAM memory structures to prevent concurrent access and ensure serial execution of kernel code. You can run SQL command to check current latch statistics data.

--Get latch statistics data
SELECT * FROM sys.dm_os_latch_stats
ORDER BY wait_time_ms desc

You can see current latch statistics details that is also not showing only real time value but it is accumulated data since last SQL Server service restarted or last time latch wait statistics was reset/cleared.

Now we will clear these latch statistics data to get real time value of latch details. Run below dbcc sqlperf command to clear the latch statistics.

DBCC SQLPERF(“sys.dm_os_latch_stats”, CLEAR)

You can see we have successfully executed above command on our SQL Server Instance.

Now go ahead and run same SQL command to get the latch statistics again. You can see now there is no waiting requests on our SQL Server Instance and you are getting real time value about latches.

Related Articles

• Should We Clear Wait Stats Data from DMV sys.dm_os_wait_stats?
• How to Fix BACKUPBUFFER Wait Type?
• How to Improve Bulk Data Load Performance?

These are the 4 use cases of DBCC SQLPERF command that we use during different database activities. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post 4 Different Usages of DBCC SQLPERF in SQL Server appeared first on Techyaz.com.

SQL Server has encountered 32 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [E:\MSSQL\Data\database_file.mdf] in database [Database_Name] (%d). The OS file handle is 0x000000F543. The offset of the latest long I/O is: 0x000000004h583.

This message indicates that SQL Server has issued a read or write request from disk, and the I/O requests taking longer than 15 seconds to return. Microsoft SQL Server reports Event Id 833 to the event log, and above text in the ERRORLOG file along with other symptoms like high wait times for PAGEIOLATCH waits or other disk IO related wait types in SQL Server.

Why we get this issue/warning?

SQL Server database is designed to perform intensive disk I/O operations. An I/O operation may be either a read or a write operation and the standard disk speed to perform these operations is around 10 milliseconds or less. If any of the read or write operation takes more time to perform their activity and generally disk speed reaches or exceeds 15 seconds of time, then database engine reports event id 833 along with above log information in error log file.

If you receive such messages, it does not mean problems with SQL Server. They are reported to let DBAs and system administrator know about poor storage performance and help them to investigate why the I/O request took so long. Long I/Os often indicate a SQL Server workload that is too intense for the disk subsystem. Disk Subsystems perform poorly/slowly because of either stuck or stalled IO operations. Before going ahead let’s first understand about stuck and stalled IO operation.

Stuck I/O

Stuck I/O is defined as an I/O request that does not finish.  To resolve a stuck I/O condition, you must usually restart the computer or perform a similar action. A stuck I/O condition typically indicates one of the following:

• Faulty hardware
• A bug in an I/O path component

Stalled I/O

Stalled I/O is defined as an I/O request that does finish or that takes excessive time to finish. Stalled I/O behavior typically occurs because of one of the following reasons:

• The hardware configuration
• The firmware settings

Below is the list of items that can cause stuck or stalled I/O operations on disk subsystem that can adversely affect SQL Server performance.

• Faulty hardware
• Incorrectly configured hardware
• Firmware settings
• Filter drivers
• Compression
• Bugs
• All transactions are in one drive like (backup, live database updates)
• Blocking
• Latch contentions
• Missing Indexes in databases

Resolution

Disk I/O problems are one of the most difficult problems to diagnose and to find the root cause. There might be multiple reasons due to which your disk subsystems perform slow. First step towards fixing such issue is to examining the event logs for any hardware related error messages and then proceed towards fix accordingly. Follow below steps to reduce the IO overheads on your database server if I/O requests taking longer than 15 seconds or perfoming slow.

1. Check & update all missing device drivers and firmware with the latest updates.

2. Analyze memory pressure and CPU usages trends as well because if you will run with CPU or memory pressure, there might be possibility slow disk performance.

3. Monitor disk I/O performance counters either using Performance Monitor or using DMVs to examine your disk subsystem workload. Read attached article to get the optimum values for all given counters. It’s highly recommended to monitor these counters for a while, and then compare it to your disk baselines value that you had captured during stress testing or load testing exercise.

• Average Disk Sec/Transfer
• Average Disk Sec/Read
• Average Disk Sec/Write
• Average Disk Queue Length
• Current Disk Queue Length
• Disk Read/Sec
• Disk Write/Sec
• %Disk Time

4. Microsoft suggests being careful when you use the performance counters because SQL Server takes full advantage of asynchronous I/O capabilities that push the disk queue lengths heavily. Therefore, longer disk queue lengths alone do not indicate a problem. So, compare every counter with their baseline and then analyze the trend.

5. Monitor SQL Server Wait Types, If the disk is slowing all the times then you will get wait types PAGEIOLATCH, WRITELOG or LOGBUFFER with high wait time.

6. Exclude SQL Server Data and Log files from antivirus software scans. If your database files are configured to be scanned by antivirus software then this will have slowed your disk performance.

7. If you are getting this issue on a specific time then I would suggest checking the conflict between your SQL Server Agent Jobs? There might be some data load/Disk IO intensive query or database maintenance activities will be running parallelly at the same time. Change the Agent jobs schedule time carefully so that they should not conflict to each other.

8. Don’t place database files and backups on single drive. Always place data file, log files and tempdb files on separate drives to reduce disk IO overhead. Also, it is highly recommended to run backups on separate drive because if you run backups on data drives then this can reduce disk performance.

9. Check your database files property for which you are getting this error. The size of database files should not be limited to fixed size with auto growth disabled. Enabling the auto growth of all the data files with appropriate value can fix this issue if you have put limits of database files growth. Read the attached article to understand the best value for database file autogrowth.

10. Disk I/O can perform poorly if there is no proper indexing on your tables. You can use Index Turning Wizard to resolve I/O pressure on the system. You can find missing indexes from most used or accessed tables and create appropriate indexes to reduce query time and resources overhead.

Disable this Error to Log using Trace flag 830

Microsoft has given us an option to disable stalled or stuck I/O detection by using trace flag 830, although we do not recommend that you do this.

To disable detection when SQL Server starts, use the -T830 startup parameter to disable detection every time that SQL Server is started. To disable detection for an instance of SQL Server that is currently running, use the following statement:

dbcc traceoff(830, -1)

This setting is effective only for the life of the SQL Server process.

Related Articles:

• Identify Disk Bottleneck in SQL Server using Perfmon Disk Counters
• How to Improve Bulk Data Load Performance?
• What is ASYNC_IO_COMPLETION wait type?
• Understanding BACKUPBUFFER wait type

I hope you like this article. Please follow our Facebook and Twitter handle to get latest updates.

The post Fix Event ID 833: SQL Server has encountered 32 occurrence(s) of I/O requests taking longer than 15 seconds to complete. appeared first on Techyaz.com.

Related Articles:

• Understanding SQL Server Wait Types
• How to deal with Resource Semaphore Wait type?
• What is BACKUPBUFFER Wait type?
• How to Reduce SQL Server Wait type PREEMPTIVE_OS_WRITEFILEGATHER?
• How to Improve SQL Server Bulk Data Load Performance?

ASYNC_IO_COMPLETION wait type occurs when a task is waiting for asynchronous I/Os to finish.  This wait type is normally seen with few other wait types like BACKUPBUFFER, BACKUPIO etc. This is clear indication of DISK IO issue. You can also get the Average disk queue length or current disk queue length value at the same time when you are getting this wait type. Compare both counters and if these counters have high value then you should look into your storage subsystem. Identify disk bottlenecks, by using Perfmon Counters, Profiler, sys.dm_io_virtual_file_stats and SHOWPLAN.

Any of the following will reduce this wait type occurrence:

1. Add additional IO bandwidth.
2. Balancing IO across other drives. If you are running your backups on same drive in which you have placed your database files. Make sure to keep separate drive for data, log and backups.
3. Reducing IO with appropriate indexing. This can cause major performance issue if you don’t have proper indexes.
4. Check for bad query plans that are consuming most of resources.
5. Check memory pressure if your server has enough memory to run resource extensive operations.

We can also correlate this wait type between Memory pressure and Disk IO subsystem issues.

If you like this tip, you can follow us on our facebook page and on Twitter handle to get latest updates.

The post Understanding ASYNC_IO_COMPLETION Wait type in SQL Server appeared first on Techyaz.com.

Today I was working on an issue in which backup job was showing in executing state since last 2 days.Initially my observation was either it was hung in middle of its process or it was processing very slow. Later, i decided to see the wait types for this process. When i was gathering this info, i saw the wait type backupbuffer for this transaction.

What is BACKUPBUFFER Wait Type

This wait type Occurs when a backup task is waiting for data, or is waiting for a buffer in which to store data. This type is not typical, except when a task is waiting for a tape mount. This wait stats will occur when you are taking the backup on the tape or any other extremely slow backup system.This wait type can be seen with one more wait type i.e. ASYNC_IO_COMPLETION.

These both wait types indicate that the issue is with storage/disk subsystem. Solution is to either ask your storage team to look in this or change your disk/storage subsystem and run your backup on a healthy and fast storage subsystem. There are other options as well where you should focus if your storage subsystem is ok. Below is the list of options you should look into to overcome from this wait type.

• Add additional IO bandwidth to accommodate the load.
• Balancing IO across other drives. If you are running your backups on same drive in which you have placed your database files. Make sure to keep separate drive for data, log and backups.
• Reducing Disk IO pressure with appropriate indexing. This can cause major performance issue if you don’t have proper indexes.
• Check for bad query plans that are consuming most of resources.
• Check memory pressure if your server has enough memory to run resource extensive operations.

Related Articles:

• Understanding SQL Server Wait Types
• What is Resource Semaphore Wait type?
• How to Improve Bulk Data Load Performance?
• Should We Clear Wait Stats Data from DMV sys.dm_os_wait_stats?

If you like this tip, you can follow us on our facebook page and on Twitter handle to get latest updates.

The post How to fix BACKUPBUFFER Wait Types? appeared first on Techyaz.com.

RESOURCE_SEMAPHORE waits occurs when a query memory request cannot be granted immediately due to other concurrent queries. High waits and wait times may indicate excessive number of concurrent queries, or excessive memory request amounts.

High waits on RESOURCE_SEMAPHORE usually result in poor response times for all database users, and need to be addressed.

It is also useful to correlate high waits on RESOURCE_SEMAPHORE with the Memory Grants Pending and Memory Grants Outstanding SQL Memory Manager performance counters. Higher values for these counters indicate a definite memory problem especially a non-zero value for Memory Grants Pending.

Resource Semaphore is a SQL Server internal mechanism that allows a query to reserve memory only if there is enough free memory available otherwise, a requesting query is forced to wait in queue. When resource semaphore receives new request, it first checks if any query is waiting or not. If any query is waiting in queue then the new request will also be added to the queue based on first come first serve allocation. All these requests in queue will show you waiting with RESOURCE_SEMAPHORE wait type. Once enough memory will be available then memory will be granted to the requests in an order they have been queued.

The root cause of this type of memory problem is when memory-intensive queries, such as those involving sorting and hashing, are queued and are unable to obtain the requested memory. The solution would be to tune the offending queries, or manage their workload so that they are executed at less busy times.

SQL Server has provided few DMVs to identify and fix such memory issues. You can have a look at DMV sys.dm_exec_query_resource_semaphore that will show the current status of resource semaphore . You can also see the output of DMV sys.dm_exec_query_memory_grants to display all queries that have memory granted and those who are waiting to be granted the required memory. All waiting queries would have null value in grant_time column in the output of DMV sys.dm_exec_query_memory_grants.

You can get the queries that are causing or eating most of the memory using above DMVs and from here you can look at the options of optimizing the memory intensive statements. You can also have a look at below article if your system is facing insufficient memory issue.

• SQL Server Error 701 and Error 802: There is Insufficient System Memory to Run this Query
• Understanding SQL Server Memory Allocation and Role of MIN/MAX Server Memory Configuration

I hope you like this article. Please drop your questions and feedbacks in below comment section. You can also follow our facebook page and Twitter handle to get latest updates.

Related Articles:

• Understanding SQL Server Wait Types
• How to Deal with ASYNC_IO_COMPLETION wait type?
• Understanding BACKUPBUFFER Wait type

The post How to Fix RESOURCE_SEMAPHORE Wait Type in SQL Server? appeared first on Techyaz.com.

Select * from sys.sysprocesses

DBCC Inputbuffer(SP ID)

DBCC SQLPERF ('sys.dm_os_wait_stats', CLEAR);
GO

The post What is SQL Server Wait Type? or How to get wait type info in sql server? appeared first on Techyaz.com.