Error: 1101, Severity: 17, State: 2
Could not allocate a new page for database ‘DBNAME’ because of insufficient disk space in filegroup ‘PRIMARY’. Create the necessary space by dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.

There is another error 1105 that is also very much like above error code. Error 1105 can also be fixed using same solution given in this article.

Error 1101 – Root Cause

Main reason behind error 1101 or error 1105 is the insufficient disk space in filegroup/files. You might also get this error if your database drives are running out of space. There are multiple reasons that can cause this error to generate. Some of them are given below.

• You have restricted database maximum file size or put size limits for files.
• Database file drives are running out of space.
• Autogrowth is not enabled for database files.

Solutions

To fix this issue we need to make space for database files to grow. Issue might vary depending on the nature of issue mentioned in above section. Sometimes, you might face this issue if maximum size for database files are restricted to some values so whenever database will grow to that point, SQL Server will not allow to allocate any page to that database. Sometimes you might get error 1101 because you don’t have any space on your database drives. Here, I will discuss each case individually and its solution to fix it.

Unrestrict or Increase Database File Size Limits

You might have enough space in your database drives but still you can get error 1101 because the maximum file size of database files is limited to some value and that causes not to grow your database beyond that size. You can unrestrict or increase the existing value of maximum file size to some bigger value. We have also an option to set database file size to unlimited size value that means database file will grow till the database file drive limit. Make sure to set appropriate autogrowth size for each database files.

You can set unlimited file size option or increase the value of maximum file size using GUI in SQL Server Management Studio or using ALTER statement.

• Connect to Database Instance in SQL Server Management Studio. Expand Database folder.
• Right Click on your database and choose Property window.
• Click on Files tab from left side pane in database property window.
• Here you can see all database files. Now you can click on three dots given next of database files in “Autogrowth/Maxsize” column. You can see max size is set to a limited value as shown in below screenshot. I have chosen unlimited option and clicked on Ok button.

Now we will not get any issue if our database drive has enough space.

Space in Database Drives

Another possibility for this error is, there might be possibility that your drives on which you have placed your database files are running out of space. If this is the case then you have below 4 options.

• Add additional database file to different disk where there is enough space.
• Move additional/unwanted files from your existing database drives.
• Drop/Purge or Move unwanted data (table/indexes)
• Add Additional Disk Space

Immediate fix of this issue is to add new database file to another drive where you have enough space to run your transactions. You can use below command to add addition database file to another disk.

USE [master]
GO
ALTER DATABASE [DBNAME]
ADD FILE (NAME = N'NewFileName', FILENAME = N'F:\mssql\data\newdatafile.ndf', SIZE = 100096KB, FILEGROWTH = 10240KB)
TO FILEGROUP [SECONDARY]
GO

You can also add new database file in SQL Server Management Studio by launching Database Property Window. Select Files from left side pane and Click on “Add” button showing in below screenshot. Make sure to set autogrowth value carefully for any database file. Also it is recommended to enable IFI – Instant File Initialization to speed up autogrowth process for your databases. Refer to below article to get best value for autogrowth for your database files. You can also read what are the benefits and how to enable IFI – Instant File Initialization in below article.

• What Should be the Best Value for database Autogrowth settings?
• How to Enable Instant File Initialization in SQL Server?

You can also have a look at the existing database drives whether there is any unwanted files or older backup files (*.bak) are not stored. Sometimes we run backups into database drives and forgot to delete or move them. Make sure to drop or move all unwanted files to some other location. Database drives should have only database files.

Final and long-term solution is the data purging. Identify the database objects that are not needed and then plan the data removal or data purging process to make space on your database.

We also have an option to extend the database drives as per the data growth. Make sure to plan this activity carefully before deploying it on production servers.

Enable Autogrowth

I have already written an article on Database autogrowth property setting. Read attached article to find out best value for Autogrowth settings.

You might get error 1101 if you have not enabled authogrowth setting for your database files. You need to enable it by launching database property window. You can click on three dots next to each database files in column “Authogrowth/Maxsize” and click at the enable authogrowth checkbox as showing in below screenshot.

You can also use ALTER statement to enable database autogrowth setting for your database file. Remember, you need to run this ALTER statement for each database file separately.

Run below ALTER command to enable autogrowth for database file.

--Pass logical filename of your database file for under NAME section.
ALTER DATABASE [Techyaz] MODIFY FILE ( NAME = N'Techyaz_Data', FILEGROWTH = 10000KB )

Make sure to enable Instant File Initialization to speedup autogrowth operations. Links to enable Instant File Initialization is attached in this article.

If you are facing this issue for tempdb database then you can consider one more option apart from all of the above options. And this option is to restart SQL Server service if it’s allowed. Keep in mind, SQL Server service restart will cause an outage for few minutes but it will create new tempdb files with initial size and error 1101 will be disappeared. This is not recommended solution because of outage so you should work on long terms solutions.

You can also use above solutions to fix error 1105 that is very much similar to error 1101. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

Read More:

• Fix Error 5144 and Error 5145 for Long or Failed Autogrow of Database Files
• Understanding Autogrowth Operation in SQL Server Databases
• How to Enable IFI – Instant File Initialization?
• Manage Database Files during huge Data Load

The post Fix Error 1101: Could not allocate a new page for database appeared first on Techyaz.com.

Msg 5144
Autogrow of file ‘Data or Log file name’ in database ‘Database’ was cancelled by user or timed out after <n> milliseconds. Use ALTER DATABASE to set a smaller FILEGROWTH value for this file or to explicitly set a new file size.

Or you will get below message if autogrow operation or data/log file size increase operation will run for longer duration than expected.

Msg 5145
Autogrow of file ‘Data or Log file name’ in database ‘Database’ took <n> milliseconds. Consider using ALTER DATABASE to set a smaller FILEGROWTH for this file.

Whenever data or log file growth operation is taking lot of time then a wait type PREEMPTIVE_OS_WRITEFILEGATHER will be shown in to SQL Server. If you want to read more about this wait type then click at the attached link. 

Root Cause

Data and log file initialization takes lot of time if you have to create large database files or large data or log file growth will be performed. Whenever we allocate or extend space to data or log file, it first fills with zeros. Zeroing operation takes lot of time for bigger size autogrowth operations that causes for longer run to complete. We generally see these messages (5144 or 5145) during either extending the data or transaction log files or during autogrowth operation for data or log files.

Microsoft has given a solution named Instant File Initialization to fix the time consuming file size allocation for data files. Once Instant File Initialization will be enabled, your data files size allocation will be done in faster way but unfortunately Instant File Initialization does not work on transaction log files.  Instant file initialization reclaims used disk space without filling that space with zeros. Instead, disk content is overwritten as new data is written to the files. Read below section to overcome these error messages.

Solution

To avoid these messages 5144 or 5145 to be logged, we should plan file size growth carefully. We should set maximum size of the database so that it could not perform frequent autogrowth operation. We can enable Instant File Initialization feature to speed up the data file size allocation. It is not recommended to rely on autogrowth of files as standard operation for obtaining more space in a file. Rather, you should plan out the maximum size of your database and/or transaction log file and allow autogrowth to occur only in unplanned situations for space. Follow below points to avoid this error.

• Enable Instant File Initialization to speed up data file size allocation. Read attached article to enable IFI (Instant File Initialization) on your database server.
• As IFI (Instant File Initialization) does not work on log files so we need to set log files in smaller chunks to avoid any time out error or longer runs.
• Set database size to the maximum size it could be and allow autogrowth settings to occur only in unplanned situations for space.Read attached article to understand Autogrowth setting and its perfect value for database files.

I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

Related Articles:

• Manage Transaction Log File during Data Load
• How to Improve Bulk Data Load Performance?
• How to Reduce SQL Server Wait type PREEMPTIVE_OS_WRITEFILEGATHER?
• Why Should You Always Turn Off Database Auto Shrink Property?
• Fix Error 3023: Shrink failed for Log file

The post Fix SQL Server Msg 5144 and 5145 for long or failed autogrow of data and log files appeared first on Techyaz.com.

Let me first give you some background about SAP migration tool like how it works. SAP Migration tool first captures every related information along with data and creates an export file that needs to be copied to the target location or server and then we perform bulk data import with the help of SAP migration tool. Bulk data import or data load will be done locally and not over network.

When we ran SAP migration tools to capture all details, a dump file was created that we have copied to the SQL Server box. We had approx. 900GB of data that needs to be loaded into SQL Server database. When we performed SQL Server data load using SAP migration tools with standard settings, it took around 14 hrs. to complete the bulk data import.  As 14 hrs. was very long time so we decided to work on reducing the total time taken by import process.

We made some changes from both sides application as well as SQL Server database side. SQL Server data load time has reduced drastically to around little more than 6 hours post these change implementations. Here, I am explaining each parameter that we have configured from database side just for this bulk data import.

NOTE: Make sure to revert these changes as per your application/vendor best practices post this bulk data import for OLTP/normal operations. We are changing value considering Data Load only.

It is important to understand the factors that affect bulk data import performance. To speed up the data load, you need to look into below configuration options from DBA standpoint.

Auto Grow

 Autogrow settings can be used to automate SQL Server database file growth. This is very useful property but this can degrade your bulk import performance or data load operations. Read attached article to understand SQL Server Autogrow setting in detail.

Autogrow setting increment must be large enough to avoid the performance penalties.  If you set default or small value during bulk import or data load, database engine will have to perform data file autogrow operation every time the file will be filled with data till maximum size limit. More autogrow operation will take more resources like IO, CPU etc and time to complete the data load transaction.

The exact value for this setting should be based on data growth. You should not be left autogrow in percentage value rather you should use some value in MBs. A general rule of thumb is to set your autogrow setting to about one-eighth the size of the file. Please read attached article to understand how to manage transaction log files during huge data load.

I would suggest keeping database size as much as you are expecting your database will grow to avoid autogrow operation. If you are aware that your database size will be 900 GB post data load, you should set your database size to 900 GB before starting the data load. This will improve the bulk import performance. Make sure to enable IFI (Instant File Initialization) for your SQL Server instance to reduce total time SQL Server takes to set the database size.

Instant File Initialization

As I suggested in above point, Turn IFI (Instant File Initialization) ON to speed up AutoGrow process. If you set a considerable large value to autogrow settings, it will take some time to increase the size by zeroing the disk address. IFI speeds up this process and allow us to create enormous size of data files in quick way. Read attached article to learn more about IFI (Instant File Initialization) and how it will speed up autogrow or data load operation. You will also learn how to enable IFI for your SQL Server instance in this attached article.

Auto Shrink

AutoShrink is very costly operation that should be always OFF even during your daily and normal operations. Turn it OFF immediately. Make sure your disk has enough space to sustain during bulk operations or data load. Repeatedly shrinking and growing the data files will cause file-system level fragmentation, which can slow down performance. It also takes lot of CPU and IO. Read attached article to know why you should always keep Auto_Shrink off.

Reduce Logging

Change recovery model to SIMPLE or BULK-LOGGED to reduce the logging operations. The simple recovery model minimally logs most bulk operations. For a database under the full recovery model, all row-insert operations that are performed during bulk import are fully logged in the transaction log.

For large data imports, this can cause the transaction log to fill rapidly. For bulk-import operations, minimal logging is more efficient than full logging and reduces the possibility that a bulk-import operation will fill the log space. To minimally log a bulk-import operation on a database that normally uses the full recovery model, you can first switch the database to the bulk-logged recovery model. After bulk importing the data, switch the recovery model back to the full recovery model.

Auto_Update_Statistics

Auto_Update_Statistics is very useful property that should be set to ON during normal business operations but here I would suggest turning it off during huge data load or bulk import operations. Updating the statistics also takes resources from the system. This can lead to performance issues when you are doing bulk data. It’s important to keep up-to-date statistics for the optimizer to use, but here we will turn it off and once data load or bulk data inserts will be performed, we will again enable it.

Auto_Create_Statistics

This should be OFF because creating statistics also takes resources from the system that can degrade the overall bulk import performance. Make sure to enable it post data load or bulk inserts operation.

Parallelism – MAXDOP

SAP suggests setting MAXDOP value to 1 during normal business operation so we had this value during first data load that took around 14 hrs. We had assigned half of the CPU cores for the parallel processes during data load and bulk import operations. I believe MAXDOP has contributed significant role in reducing total data load time. Make sure to revert this change to its original value as per your application best practices.

You can change MAXDOP settings either using sp_configure command or using GUI in SSMS by launching SQL Server Instance property window.

Using Batches

Make sure to use an optimum batch size that will increase the bulk data import performance. Importing a large set of data as a single batch can be problematic, so bcp and BULK INSERT let you import data in a series of batches, each of which is smaller than the data file. Each batch is imported and logged in a separate transaction, and after a given transaction is committed, the rows imported by that transaction are committed. If the operation fails, only rows imported from the current batch are rolled back, and you can resume importing data starting at the beginning of the failed batch rather than at the beginning of the data file.

I hope you like this article. Please follow us our Facebook page and Twitter handle to get latest updates.

The post How to Improve SQL Server Bulk Data Import Performance? appeared first on Techyaz.com.

Set Optimum Database Size

Your database should have enough size that it never grows and if it grows make sure that Instant file initialization should be enabled on that SQL Server Instance. Autogrowth is an option that should be used to avoid unexpected scenarios of space issues in database files. You should plan your database size in a way that can minimize the use of autogrowth setting. You should figure out how big your database will be over the time and set the database size to that value. Analyze database size on every few months to adjust the size if it requires to increase. This way you can minimize autogrow events to be performed on your database files.

Resize Database Autogrowth in SQL Server

Auto growth events are expensive operations that slow down the performance of your database because whenever an auto-growth event is performed, SQL Server holds up database processing. This equates to slower response time for those SQL commands that are being processing against the database that is growing.

If you run a large transaction that requires the log to grow, other transactions that require a write to the transaction log will also have to wait until the autogrow operation completes. If your database is performing frequent autogrow events, then you have a good scope of performance improvement by proper resizing the database autogrowth settings that will reduce the number of autogrow events to be occurred.

Another negative side of autogrowth is the disk fragmentation. The more autogrow events you have the more physical fragmentation you will have. When your database is physically fragmented, it will take more time for SQL Server to read that databases. To avoid the issues associated with auto-growth events you need to minimize the number of its occurrence.

Change Default Autogrowth Settings

As per my experience, never leave your database files with the default autogrowth settings. The autogrowth default value for data files is 1MB and log files is 10% of total size of the log file. These autogrowth values are inherited from model database. You can change the autogrowth size of each database file of model database that will apply on all databases we create on this SQL Server Instance.

For example: You can change autogrow settings from default value that is 1MB to one-eighth of the database file size. This will again reduce the total number of autogrow events of the database. Make sure that Instant file initialization should be enabled on that SQL Server Instance.

Monitoring

If you know your database growth pattern, then set autogrowth per that growth. If you don’t have any idea about your future database growth, then you should proactively monitor database file sizes and their autogrowth events for a week or month and then set it per the growth pattern. We should also proactively monitor database growth histories for all databases to avoid any future outages due to disk or database file full.

You can analyze database disk usage dashboard report to get the number of times autogrowth event has occurred for your database files. I have explained this later in this article.

Best Practices for Database Autogrowth in SQL Server

You can improve the database performance and can manage the disk space utilization by focusing on below four points. This will also reduce the number of Autogrowth events to be occurred in the database files.

1. Analyze your database growth pattern and set optimum size of database with the analyzed growth settings. Your database should have enough size that it never grows and if it grows make sure that Instant file initialization should be enabled on that SQL Server Instance.  This will drastically reduce the SQL Server Autogrowth events in day to day life.
2. Proactively monitor database file sizes and their autogrowth events. This helps you to analyze the growth pattern of database files and avoid fragmentation.
3. Consider defragmenting your database file system if you have lot of auto-growth events occurred on your databases.
4. Never leave your database autogrowth with default values, change it as per growth pattern that you analyzed over time.
5. Avoid using autogrowth in percentage, rather give a specific amount of size in MB/GB. Microsoft suggests to set your autogrow setting to about one-eighth the size of the file and test it before deploying on PROD.
6. Turn on the <MAXSIZE> setting for each file to prevent any one file from growing to a point where it uses up all available disk space.
7. The growth increment of your transaction log must be large enough to stay ahead of the needs of your transaction units. Even with autogrow turned on, you can receive a message that the transaction log is full, if it cannot grow fast enough to satisfy the needs of your query.

Change Database Autogrowth in SQL Server?

You can run below command to change the autogrowth value of any database file.

--Pass logical filename of your database file for under NAME section.
ALTER DATABASE [Techyaz] MODIFY FILE ( NAME = N'Techyaz_Data', FILEGROWTH = 10000KB )

You can also change the autogrowth settings using GUI in SSMS. Launch the database property window. Click on files tab from left side pane and then on the three dots (…) that are showing besides every database file. You can see this window in below screenshot.

How to View Total Number of Autogrow Events Occurred on a Database?

You can find the number of autogrowth events occurred for your database by running “Disk Usage” dashboard report. Right click on your database for which you want to see total number of autogrowth events. Choose “Reports” and then select “Disk Usage” as shown in below image.

You will get below dashboard report for disk usage. You can see all details regarding space usage. Auto growth events are also captured in the same report. I have highlighted those entries in below screenshot.

Once you expend the plus + sign, you can see total number of autogrow events that have been occurred for your database. You can count the total numbers and total space as well. Based on this analysis you can alter your autogrowth settings to the optimum value that will reduce that much numbers as it is showing here.

I hope you like this article. Please follow us on our facebook page and on Twitter handle to get latest updates.

Reference: MSDN Article based on Autogrowth

The post Understanding Database Autogrowth in SQL Server appeared first on Techyaz.com.