Msg 5144
Autogrow of file ‘Data or Log file name’ in database ‘Database’ was cancelled by user or timed out after <n> milliseconds. Use ALTER DATABASE to set a smaller FILEGROWTH value for this file or to explicitly set a new file size.

Or you will get below message if autogrow operation or data/log file size increase operation will run for longer duration than expected.

Msg 5145
Autogrow of file ‘Data or Log file name’ in database ‘Database’ took <n> milliseconds. Consider using ALTER DATABASE to set a smaller FILEGROWTH for this file.

Whenever data or log file growth operation is taking lot of time then a wait type PREEMPTIVE_OS_WRITEFILEGATHER will be shown in to SQL Server. If you want to read more about this wait type then click at the attached link. 

Root Cause

Data and log file initialization takes lot of time if you have to create large database files or large data or log file growth will be performed. Whenever we allocate or extend space to data or log file, it first fills with zeros. Zeroing operation takes lot of time for bigger size autogrowth operations that causes for longer run to complete. We generally see these messages (5144 or 5145) during either extending the data or transaction log files or during autogrowth operation for data or log files.

Microsoft has given a solution named Instant File Initialization to fix the time consuming file size allocation for data files. Once Instant File Initialization will be enabled, your data files size allocation will be done in faster way but unfortunately Instant File Initialization does not work on transaction log files.  Instant file initialization reclaims used disk space without filling that space with zeros. Instead, disk content is overwritten as new data is written to the files. Read below section to overcome these error messages.

Solution

To avoid these messages 5144 or 5145 to be logged, we should plan file size growth carefully. We should set maximum size of the database so that it could not perform frequent autogrowth operation. We can enable Instant File Initialization feature to speed up the data file size allocation. It is not recommended to rely on autogrowth of files as standard operation for obtaining more space in a file. Rather, you should plan out the maximum size of your database and/or transaction log file and allow autogrowth to occur only in unplanned situations for space. Follow below points to avoid this error.

• Enable Instant File Initialization to speed up data file size allocation. Read attached article to enable IFI (Instant File Initialization) on your database server.
• As IFI (Instant File Initialization) does not work on log files so we need to set log files in smaller chunks to avoid any time out error or longer runs.
• Set database size to the maximum size it could be and allow autogrowth settings to occur only in unplanned situations for space.Read attached article to understand Autogrowth setting and its perfect value for database files.

I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

Related Articles:

• Manage Transaction Log File during Data Load
• How to Improve Bulk Data Load Performance?
• How to Reduce SQL Server Wait type PREEMPTIVE_OS_WRITEFILEGATHER?
• Why Should You Always Turn Off Database Auto Shrink Property?
• Fix Error 3023: Shrink failed for Log file

The post Fix SQL Server Msg 5144 and 5145 for long or failed autogrow of data and log files appeared first on Techyaz.com.

What is PREEMPTIVE_OS_WRITEFILEGATHER?

As per MSDN, PREEMPTIVE_OS_WRITEFILEGATHER is a wait type that occurs while SQL Server is waiting for the Operating System to complete portions of write operations. This wait is commonly seen in databases with auto-growth not configured appropriately or during large size allocation to database files. If you are getting this wait type frequently then you should investigate your SQL Server instance. You can also observe this wait type along with BACKUPBUFFER  and LOGBUFFER wait types.

Sometimes, we get this wait type during database restore operation as well. As we know backup operation captures details like database file sizes irrespective of how much data or logs filled inside the data or log file. When you execute restore operation, it will create that much big file and it will take more time because SQL Server will be busy in zeroing out the file equivalent to source data or log file size. If you have enabled IFI on your database server then you will not be getting this wait type for data files but still you might get it for log file size allocation.

Suppose your database has 100GB of log file and only 12 GB is filled. If you take backup of this database and restore it anywhere then restore operation will create a log file of 100GB that will take lot much time because of zero initialized operation to be performed for all 100GB size.

Solutions to Reduce Wait Type PREEMPTIVE_OS_WRITEFILEGATHER

Microsoft has given a feature Instant File Initialization to overcome or reduce this wait type but this feature works for only data files not log files. We hope this feature will work on both data as well as log files in future releases of SQL Servers.

If you are getting this wait type during extending data files or during data file autogrowth then you can simply enable Instant File Initialization on your SQL Server database server and this wait type will disappear from next time. This feature will skip zero-initialization of data files and directly allocate given size to the data files. Read attached article to learn Instant File Initialization and how to enable it on your SQL Server Instance?

But we know this feature does not work on log files so we cannot use this feature to reduce this wait type if it occurs during extending log file or during log file autogrowth. We faced same issue because we had already enabled Instant File Initialization for data files but stuck during extending log files. Here I will give you few best practices that you can apply to avoid and reduce this wait type.

1. Increase auto-growth from the default value to some numeric value to reduce the frequent auto-growth occurrence for your database file. Thumb rule is to keep autogrowth size one eight of the corresponding database-file size. Read this article to know the Perfect value for database file Autogrowth?
2. If possible avoid auto-growth occurrence by allocating expected size grown in future to the database.
3. Monitor Storage subsystems and Investigate if any issues identified. Upgrade Storage if required.
4. Have a look at the Windows Event Viewer for any system related issues.
5. Enable Instant File Initialization to avoid zero-initialization for data files.
6. Consolidate transactions from row-by-row statements to batch processing.
7. If you have bigger log file having small number of logs filled inside. You can check log file size using DBCC SQLPERF command given in attached article. You can shrink the log file and take the backup again to perform the restore. This should be a quick process rather than waiting for zeroing to complete for previous size of log file.
8. Stop any activity to free-up the target disk’s header so that it can dedicatedly work on zeroing out the LDF file.

Read More:

• How to Speed up Huge Data Load in SQL Server?
• Understanding ASYNC_IO_COMPLETION Wait Type
• Manage Transaction Log File during Data Load
• Understanding SQL Server Wait Types
• How to reduce RESOURCE_SEMAPHORE wait type?

I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

The post How to Reduce PREEMPTIVE_OS_WRITEFILEGATHER Wait Type in SQL Server? appeared first on Techyaz.com.

Let me first give you some background about SAP migration tool like how it works. SAP Migration tool first captures every related information along with data and creates an export file that needs to be copied to the target location or server and then we perform bulk data import with the help of SAP migration tool. Bulk data import or data load will be done locally and not over network.

When we ran SAP migration tools to capture all details, a dump file was created that we have copied to the SQL Server box. We had approx. 900GB of data that needs to be loaded into SQL Server database. When we performed SQL Server data load using SAP migration tools with standard settings, it took around 14 hrs. to complete the bulk data import.  As 14 hrs. was very long time so we decided to work on reducing the total time taken by import process.

We made some changes from both sides application as well as SQL Server database side. SQL Server data load time has reduced drastically to around little more than 6 hours post these change implementations. Here, I am explaining each parameter that we have configured from database side just for this bulk data import.

NOTE: Make sure to revert these changes as per your application/vendor best practices post this bulk data import for OLTP/normal operations. We are changing value considering Data Load only.

It is important to understand the factors that affect bulk data import performance. To speed up the data load, you need to look into below configuration options from DBA standpoint.

Auto Grow

 Autogrow settings can be used to automate SQL Server database file growth. This is very useful property but this can degrade your bulk import performance or data load operations. Read attached article to understand SQL Server Autogrow setting in detail.

Autogrow setting increment must be large enough to avoid the performance penalties.  If you set default or small value during bulk import or data load, database engine will have to perform data file autogrow operation every time the file will be filled with data till maximum size limit. More autogrow operation will take more resources like IO, CPU etc and time to complete the data load transaction.

The exact value for this setting should be based on data growth. You should not be left autogrow in percentage value rather you should use some value in MBs. A general rule of thumb is to set your autogrow setting to about one-eighth the size of the file. Please read attached article to understand how to manage transaction log files during huge data load.

I would suggest keeping database size as much as you are expecting your database will grow to avoid autogrow operation. If you are aware that your database size will be 900 GB post data load, you should set your database size to 900 GB before starting the data load. This will improve the bulk import performance. Make sure to enable IFI (Instant File Initialization) for your SQL Server instance to reduce total time SQL Server takes to set the database size.

Instant File Initialization

As I suggested in above point, Turn IFI (Instant File Initialization) ON to speed up AutoGrow process. If you set a considerable large value to autogrow settings, it will take some time to increase the size by zeroing the disk address. IFI speeds up this process and allow us to create enormous size of data files in quick way. Read attached article to learn more about IFI (Instant File Initialization) and how it will speed up autogrow or data load operation. You will also learn how to enable IFI for your SQL Server instance in this attached article.

Auto Shrink

AutoShrink is very costly operation that should be always OFF even during your daily and normal operations. Turn it OFF immediately. Make sure your disk has enough space to sustain during bulk operations or data load. Repeatedly shrinking and growing the data files will cause file-system level fragmentation, which can slow down performance. It also takes lot of CPU and IO. Read attached article to know why you should always keep Auto_Shrink off.

Reduce Logging

Change recovery model to SIMPLE or BULK-LOGGED to reduce the logging operations. The simple recovery model minimally logs most bulk operations. For a database under the full recovery model, all row-insert operations that are performed during bulk import are fully logged in the transaction log.

For large data imports, this can cause the transaction log to fill rapidly. For bulk-import operations, minimal logging is more efficient than full logging and reduces the possibility that a bulk-import operation will fill the log space. To minimally log a bulk-import operation on a database that normally uses the full recovery model, you can first switch the database to the bulk-logged recovery model. After bulk importing the data, switch the recovery model back to the full recovery model.

Auto_Update_Statistics

Auto_Update_Statistics is very useful property that should be set to ON during normal business operations but here I would suggest turning it off during huge data load or bulk import operations. Updating the statistics also takes resources from the system. This can lead to performance issues when you are doing bulk data. It’s important to keep up-to-date statistics for the optimizer to use, but here we will turn it off and once data load or bulk data inserts will be performed, we will again enable it.

Auto_Create_Statistics

This should be OFF because creating statistics also takes resources from the system that can degrade the overall bulk import performance. Make sure to enable it post data load or bulk inserts operation.

Parallelism – MAXDOP

SAP suggests setting MAXDOP value to 1 during normal business operation so we had this value during first data load that took around 14 hrs. We had assigned half of the CPU cores for the parallel processes during data load and bulk import operations. I believe MAXDOP has contributed significant role in reducing total data load time. Make sure to revert this change to its original value as per your application best practices.

You can change MAXDOP settings either using sp_configure command or using GUI in SSMS by launching SQL Server Instance property window.

Using Batches

Make sure to use an optimum batch size that will increase the bulk data import performance. Importing a large set of data as a single batch can be problematic, so bcp and BULK INSERT let you import data in a series of batches, each of which is smaller than the data file. Each batch is imported and logged in a separate transaction, and after a given transaction is committed, the rows imported by that transaction are committed. If the operation fails, only rows imported from the current batch are rolled back, and you can resume importing data starting at the beginning of the failed batch rather than at the beginning of the data file.

I hope you like this article. Please follow us our Facebook page and Twitter handle to get latest updates.

The post How to Improve SQL Server Bulk Data Import Performance? appeared first on Techyaz.com.

Read more about auto growth and auto shrink operation in SQL Server. Both are very important activity you should review to improve your database performance.

• Understanding Database Autogrowth in SQL Server
• Why Should You Always Turn Off Database Auto Shrink Property?

What is Instant File Initialization?

Data and log file initialization takes lot of time if you have to create large database files. This can also increase total completion time of large database restore or large amount of data import to the SQL Server. Microsoft has given a feature called Instant file initialization that allows fast data file allocations for all data file operations. This feature reclaims used disk space without filling that space with zeros. Instead, disk content is overwritten as new data is written to the files. But unfortunately, this feature doesn’t work with log files so they cannot be initialized instantaneously. This is not enterprise feature any SQL Server edition can use it.

As we know DELETE operation deallocates the space and a new allocation can reuse that space which may still have data in it. When we do not zero out the existing space in case of instant file initialization there is a possibility that someone could read data that you thought had been deleted. This is a known security risk of this feature that must be considered while enabling it for your SQL Server Instance. Instant File Initialization is not enabled by default in SQL Server because of this security risk.

Enable Instant File Initialization for SQL Server?

Below are the steps you need to perform to enable Instant File Initialization.

Step 1- Login to server where you want to enable Instant File Initialization. This might be your database server or backup server.
Step 2- Open the Local Security Policy application. You can go to Control Panel->Administrative Tools->Local Security Policy. You can also open it directly by putting secpol.msc into run command prompt.

Step 3- In the left pane, expand Local Policies, and then click User Rights Assignment.

Step 4- In the right pane, double-click Perform volume maintenance tasks.

Step 5- Click Add User or Group and add any user accounts that are used to run SQL Server services. You can get SQL Server service account from SQL Server Configuration Manager or from services.msc console.

Now add account Domain\techyaz to above policy. It would be great if you create a group and add this account to that group and then add that group to given policy so that every time you don’t need to add user account here if SQL Server service account got changed.

Step 6- Once SQL Server service account will be added, it will look like below image. Click Apply, and then close all Local Security Policy dialog boxes.

Step 7- Restart SQL Server services to apply this change.

If you need to enable this feature on Cluster environment, make sure to perform this activity on all cluster nodes. If you are running with multiple SQL Server Instances, add all required security logins/group to this policy.

How to Check If instant file initialization is Enabled Or Not?

Now if you think how to know Instant File Initialization is enabled on your SQL Server Instance then here is the answer. You should check SQL Server logs to get the details about Instant File Initialization. When SQL Server service starts, database engine checks all details of SQL Server configurations and capture them into log file. You can see my error log file after restarting SQL Server service. Instant File Initialization is showing as enabled on this server.

2017-06-16 11:16:49.15 Server Database Instant File Initialization: enabled. For security and performance considerations see the topic ‘Database Instant File Initialization’ in SQL Server Books Online. This is an informational message only. No user action is required.

Advantages

Instant File Initialization improves performance of multiple database activities where large size of data file creation or Autogrow operation will perform. This can vastly speed up these operations as zero-initialization can take a long time for large files. Below are the operations that can take advantage from this feature.

• Data Load/Import
• New Database Creation
• Database Restore
• Autogrow Operation
• Data file size Increase
• Backup file Creation

Limitations

This feature will not work if:

• Your file is a log file.
• The account has not the required privilege.
• It is a sparse file – Database Snapshots.
• Transparent Data Encryption (TDE) is enabled.
• Trace flag 1806 is enabled to disable instant file initialization.

I hope you like this article. Please follow us on our facebook page and on Twitter handle to get latest updates.

Read More: 

• SQL Server Error 5009: One or more files listed in the statement could not be found or could not be initialized
• Fix SQL Server Msg 5144 and 5145 for long or failed autogrow of data and log files
• Manage Transaction Log File during Data Load

The post What is Instant File Initialization and How to enable it on SQL Server Instance? appeared first on Techyaz.com.