If your drive letters or folder structures are identical on all replica servers then most of the time you will not face any issue while adding the database files but sometimes if you mount additional drive on all replicas to make it identical and then add data file in this new drive then you might face the issue. If all replica servers don’t have identical path then adding database file operation on primary replica will not successfully replicate to other replica servers because of different drive and folder structure. This, in turn, causes the secondary replica to enter the NOT SYNCHRONIZING state.

Note: Always try to keep drive and folder structure identical on all secondary replica servers to the corresponding primary replica.

If you have added a data file on primary database of Always on Availability group then you will see multiple errors logged in SQL Server error log on your secondary replica where secondary database is suspended from always on availability group and showing in “Not Synchronizing” state. You might see one of the below error codes in your log file.

Error:5123, Severity:16, State:1.
CREATE FILE encountered operating system error 3(The system cannot find the path specified.) while attempting to open or create the physical file ‘E:\MSSQL\Data\Techyaz_Data2.ndf’.

Or

Error:5183, Severity:16, State:1.
Cannot create the file “Techyaz_Data2”. Use WITH MOVE to specify a usable physical file name. Use WITH REPLACE to overwrite an existing file.

Or

Error: 3313, Severity: 21, State: 2.
During redoing of a logged operation in database ‘Techyaz’, an error occurred at log record ID (76:659:1).

Now i will describe how to fix such issues or how to plan and add a database file to always on availability databases.

Add Data file to Availability Database

If you have already added data file on primary database and your secondary databases are suspended from availability group then we need to fix it to reestablish always on availability group. If you have not added the data file and want to add it then you can keep reading this article as i have described it later in this article. I have described how to plan database file addition to avoid any future synchronization issue.

Let’s go ahead with the first option where you have already added the data file to primary availability database and secondary database status is showing as not synchronizing due to this change. Now we need to fix this issue to reestablish the synchronization between replicas of availability group. To resolve this problem the DBAs  must complete the following steps:

1. Remove the secondary database from the availability group that is showing as “Not Synchronizing”. To do this, either you can use GUI method in SSMS or run below T-SQL statement. You can connect to secondary replica then you can expand Always On High Availability node followed by Availability Groups Now, select the availability group and then expand the Availability Databases node. To remove a single database, select it in either the Object Explorer pane or the Object Explorer Details pane. Right-click the selected database or databases, and select Remove Secondary Database in the command menu. If you want to do it using T-SQL statement then you must connect to secondary replica and execute below Alter command.

ALTER DATABASE Techyaz SET HADR OFF;  
GO 

2. Once database will be removed from AOAG, restore a full backup of the filegroup that contains the added file to the secondary database, using WITH NORECOVERY and WITH MOVE option. This option is mandatory if you don’t have any log backups. Although sometimes you can do it by only running log backup on primary replica and restore it on secondary replica in norecovery mode as mentioned in next step.
3. Now, back up the transaction log that contains the add-file operation on the primary database, and manually restore the log backup on the secondary database using WITH NORECOVERY and WITH MOVE.
4. Prepare the secondary database for re-joining the availability group, by restoring, WITH NO RECOVERY, any other outstanding log backups from the primary database.
5. Rejoin the secondary database to the availability group. Read this article to get step by step method to add a database to availability group.

If you havenot added the data file then you can plan it with following steps.

1. Remove the database from secondary replica as described in above section. Now, secondary database will be showing in to restoring state.
2. Add the data file to your availability database on primary replica.
3. Take a log backup of this availability database on primary replica.
4. Copy this log backup to secondary replica and restore it on its corresponding secondary replica using NORECOVERY and WITH MOVE option.
5. Now add the database back to always on Availability Group.

Here, I have shown you step by step process to add data file to availability database. I have also let you know how to fix the issues that might arise due to adding the database files to availability database on primary replica. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

Read More:

• Upgrade or Patch SQL Server Failover Cluster Instance Running with Availability Group
• Understanding Enhanced Database Failover in Always on Availability Group
• How to Change Failover Mode of Availability Replica in AOAG?
• Always ON Availability Group Interview Questions & Answers

The post How to Add Data File to Availability Database? appeared first on Techyaz.com.

Microsoft SQL Server Error 5009 While Adding Database File

I was working on a migration project where we had to load huge data in a SQL Server database on the target server. To optimize data load performance, we had decided to create database with the final size on the destination server. The final size of database post data import was around 800 GB. We made some configurational changes to reduce data import time. Extending database files size was one of the step to reduce data load time. If you want to learn how to speed up data import then I would suggest reading attached article.

We decided to add one more log file on separate drive considering extensive log growth during data load. While adding the log file, we got error 5009. Full error details are given below.

One or more files listed in the statement could not be found or could not be initialized. CREATE FILE encountered operating system error 5(Access is denied.) while attempting to open or create the physical file ‘T:\Techyaz_log3.ldf’.(Microsoft SQL Server, Error: 5009)

As error suggests, I don’t had enough permission to create this file on given location where i was trying to create the log file. There are two solutions to fix this issue. First solution is to set the Operating system permission on the mentioned .ldf file to full permission for your login account and SQL Server service account. While second solution is to change the location to the path where you have enough permission to create or add database files. Any of these two solutions will fix your problem. Let’s go to another version of Microsoft SQL Server Error 5009 while extending the log file.

Microsoft SQL Server Error 5009 While Extending the Size of Log File

When i changed the location to create this ldf file to a directory T:\Data\Techyaz_log3.ldf this time log file was created but we get same issue again while extending the size of this log file. Next, I was assigning bigger value to each data and log files. We had successfully increased data files size in few minutes because we had already enabled Instant File Initialization to skip zeros process on storage but we faced Microsoft SQL Server error 5009 again while increasing the size of log files. i was extending  log file with bigger values. As we know Instant File Initialization does not work for transaction log files so extending the size of log files was taking lot of time. This time access denied issue was disappeared but error 5009 issue was there only. Error description for this error is given below.

Msg 5009, Level 16, State 9, Line 1
One or more files listed in the statement could not be found or could not be initialized.

To fix this error, we had passed small value to extend the file size. Earlier we were trying to increase the log file in a slot of 40 GB in single go. It was taking time to finish but most of time log file size got extended with 40GB but sometimes we had received above Microsoft SQL Server error 5009. When we got this error, we extend the size of log file to 10Gb in place of 40GB that we were doing previously. Reducing the value while extending the log file has fixed this issue and log files had been extended to its future size successfully.

Microsoft SQL Server Error 5009 While Removing Database File

Some of the blogs have suggested that they got SQL Server error 5009 while removing a logical file in SQL Server but surprisingly I got this error while creating or increasing the size of log file. So if you are getting this error while removing any database file then you can keep reading this article to solve this issue.

Whenever we add any database file then its entry got captured in system catalog files, if you are trying to delete any database file then there might be possibility that logical name of the file that you try to remove is not unique in the system catalog tables. For example, this issue occurs if the file existed in the database earlier, and then the file was removed. When you remove the file, it will try to drop earlier file that has been removed but its metadata is not cleaned and appear like ghost file.

You should follow below steps to fix this issue.

• Create a new logical file with different name and the same data type. For example, If your file;s logical name is Techyaz_Logical_Name then create this new file with “Techyaz_Logical_Name_New”. Run below command to create this file.

ALTER DATABASE [Techyaz] 
ADD FILE ( NAME = N'Techyaz_Logical_Name_New', FILENAME = N'T:\DATA\Techyaz_file4.ndf', SIZE = 10MB, MAXSIZE = 100MB)

• Now we will remove the logical file that we have created in step 1, as in the following example:

ALTER DATABASE [Techyaz] REMOVE FILE [Techyaz_Logical_Name_New]

• Run a transaction log backup of the database.
• Now go ahead and remove the logical file that were trying to remove earlier and your received Microsoft SQL Server Error 5009.

Here, I have described multiple possibilities and fixes for Microsoft SQL Server Error 5009. I hope you like this article. Please follow us on our facebook page and on Twitter handle to get latest updates.

Read More:

• Manage Transaction Log File during Data Load
• How to get total no of virtual log files in a transaction log file
• Fix Error 3023: Shrink failed for Log file
• Understanding SQL Server Transaction Log Architecture
• Fix SQL Server Error 1105: Could Not Allocate Space for Object in Database because the Filegroup is Full
• Fix SQl Server Error 5144 and 5145 for long and failed Autogrow of data and log file

The post SQL Server Error 5009: One or more files listed in the statement could not be found or could not be initialized appeared first on Techyaz.com.

One thing we should always keep in mind that if we change database default path in SQL Server then it will not change or move the database file locations for your existing databases. If you want to move all existing databases or you want to change database files path to the new location then you need to manually do it for each database.

Once you will change default database file locations then all databases you will create on that SQL Server Instance will be created on new location. Suppose your existing SQL Server default database location is set in C drive and now you have decided to change it to D drive for data files and F drive for Log files then all databases you will create post this change will be created in these both drives. If you will mention any other specific drive during database creation then SQL Server uses that drive and not these database default locations. The SQL Server data file will be created in D drive and log file will be created in F drive but all the existing databases that were created earlier in C drive will not move to new SQL server database file location. If you want to move these databases or want to change database files location then you need to manually do it either by detach-attach method or by ALTER statement.

Change Database Default Locations in SQL Server

It’s very simple method to change default log file location or default data file location in SQL Server. Let’s start to change default database path in SQL Server.

• Launch SQL Server Management Studio and connect to your SQL Server Instance for which you must change default log file location or default data file location in SQL Server.
• Now right click at SQL Server Instance node and choose Properties option.
• You will get server properties window on your screen. Click at “Database Settings” from left side pane. You will get all server level details for SQL Server databases in right side pane. Look at the database default locations option given bottom of the right-side pane. Here you can see separate default location for data files, Log file and backup files.

• You can see database default locations for each file are showing in C drive. It means every database that will be created without mentioning their data or log file location will go to above given SQL Server database default locations.
• To change database default path, you need to browse current database default locations for each file to new the new location. Click at three dots showing next to data file or log file locations and then select the new path.
• Repeat same exercise for each file. To change default data file location, click at three dots next to data path and select new path here. Similarly, click at three dots next to log file location to change default log file location. If you want to change default backup file location then do the same for backup location as well.
• Click ak Ok button once you made changes for desired database file locations.
• Last step is to restart SQL Server services to apply this change and bring it into effect.

Once you will restart SQL Server services, your SQL Server default database location will be changed to the new path. I hope you like this article. Please follow our Facebook page and Twitter handle to get latest updates.

Read More:

• Get Total Number of Virtual Log Files in SQL Server Transaction Log File
• SQL Server Error 1105: Could Not Allocate Space for Object in Database because the filegroup is full
• How to Change SQL Server Instance Name?
• Why Should You Always Turn Off Database Auto Shrink Property?

The post How to Change Database Default Locations in SQL Server? appeared first on Techyaz.com.

Let me first give you some background about SAP migration tool like how it works. SAP Migration tool first captures every related information along with data and creates an export file that needs to be copied to the target location or server and then we perform bulk data import with the help of SAP migration tool. Bulk data import or data load will be done locally and not over network.

When we ran SAP migration tools to capture all details, a dump file was created that we have copied to the SQL Server box. We had approx. 900GB of data that needs to be loaded into SQL Server database. When we performed SQL Server data load using SAP migration tools with standard settings, it took around 14 hrs. to complete the bulk data import.  As 14 hrs. was very long time so we decided to work on reducing the total time taken by import process.

We made some changes from both sides application as well as SQL Server database side. SQL Server data load time has reduced drastically to around little more than 6 hours post these change implementations. Here, I am explaining each parameter that we have configured from database side just for this bulk data import.

NOTE: Make sure to revert these changes as per your application/vendor best practices post this bulk data import for OLTP/normal operations. We are changing value considering Data Load only.

It is important to understand the factors that affect bulk data import performance. To speed up the data load, you need to look into below configuration options from DBA standpoint.

Auto Grow

 Autogrow settings can be used to automate SQL Server database file growth. This is very useful property but this can degrade your bulk import performance or data load operations. Read attached article to understand SQL Server Autogrow setting in detail.

Autogrow setting increment must be large enough to avoid the performance penalties.  If you set default or small value during bulk import or data load, database engine will have to perform data file autogrow operation every time the file will be filled with data till maximum size limit. More autogrow operation will take more resources like IO, CPU etc and time to complete the data load transaction.

The exact value for this setting should be based on data growth. You should not be left autogrow in percentage value rather you should use some value in MBs. A general rule of thumb is to set your autogrow setting to about one-eighth the size of the file. Please read attached article to understand how to manage transaction log files during huge data load.

I would suggest keeping database size as much as you are expecting your database will grow to avoid autogrow operation. If you are aware that your database size will be 900 GB post data load, you should set your database size to 900 GB before starting the data load. This will improve the bulk import performance. Make sure to enable IFI (Instant File Initialization) for your SQL Server instance to reduce total time SQL Server takes to set the database size.

Instant File Initialization

As I suggested in above point, Turn IFI (Instant File Initialization) ON to speed up AutoGrow process. If you set a considerable large value to autogrow settings, it will take some time to increase the size by zeroing the disk address. IFI speeds up this process and allow us to create enormous size of data files in quick way. Read attached article to learn more about IFI (Instant File Initialization) and how it will speed up autogrow or data load operation. You will also learn how to enable IFI for your SQL Server instance in this attached article.

Auto Shrink

AutoShrink is very costly operation that should be always OFF even during your daily and normal operations. Turn it OFF immediately. Make sure your disk has enough space to sustain during bulk operations or data load. Repeatedly shrinking and growing the data files will cause file-system level fragmentation, which can slow down performance. It also takes lot of CPU and IO. Read attached article to know why you should always keep Auto_Shrink off.

Reduce Logging

Change recovery model to SIMPLE or BULK-LOGGED to reduce the logging operations. The simple recovery model minimally logs most bulk operations. For a database under the full recovery model, all row-insert operations that are performed during bulk import are fully logged in the transaction log.

For large data imports, this can cause the transaction log to fill rapidly. For bulk-import operations, minimal logging is more efficient than full logging and reduces the possibility that a bulk-import operation will fill the log space. To minimally log a bulk-import operation on a database that normally uses the full recovery model, you can first switch the database to the bulk-logged recovery model. After bulk importing the data, switch the recovery model back to the full recovery model.

Auto_Update_Statistics

Auto_Update_Statistics is very useful property that should be set to ON during normal business operations but here I would suggest turning it off during huge data load or bulk import operations. Updating the statistics also takes resources from the system. This can lead to performance issues when you are doing bulk data. It’s important to keep up-to-date statistics for the optimizer to use, but here we will turn it off and once data load or bulk data inserts will be performed, we will again enable it.

Auto_Create_Statistics

This should be OFF because creating statistics also takes resources from the system that can degrade the overall bulk import performance. Make sure to enable it post data load or bulk inserts operation.

Parallelism – MAXDOP

SAP suggests setting MAXDOP value to 1 during normal business operation so we had this value during first data load that took around 14 hrs. We had assigned half of the CPU cores for the parallel processes during data load and bulk import operations. I believe MAXDOP has contributed significant role in reducing total data load time. Make sure to revert this change to its original value as per your application best practices.

You can change MAXDOP settings either using sp_configure command or using GUI in SSMS by launching SQL Server Instance property window.

Using Batches

Make sure to use an optimum batch size that will increase the bulk data import performance. Importing a large set of data as a single batch can be problematic, so bcp and BULK INSERT let you import data in a series of batches, each of which is smaller than the data file. Each batch is imported and logged in a separate transaction, and after a given transaction is committed, the rows imported by that transaction are committed. If the operation fails, only rows imported from the current batch are rolled back, and you can resume importing data starting at the beginning of the failed batch rather than at the beginning of the data file.

I hope you like this article. Please follow us our Facebook page and Twitter handle to get latest updates.

The post How to Improve SQL Server Bulk Data Import Performance? appeared first on Techyaz.com.

Read more about auto growth and auto shrink operation in SQL Server. Both are very important activity you should review to improve your database performance.

• Understanding Database Autogrowth in SQL Server
• Why Should You Always Turn Off Database Auto Shrink Property?

What is Instant File Initialization?

Data and log file initialization takes lot of time if you have to create large database files. This can also increase total completion time of large database restore or large amount of data import to the SQL Server. Microsoft has given a feature called Instant file initialization that allows fast data file allocations for all data file operations. This feature reclaims used disk space without filling that space with zeros. Instead, disk content is overwritten as new data is written to the files. But unfortunately, this feature doesn’t work with log files so they cannot be initialized instantaneously. This is not enterprise feature any SQL Server edition can use it.

As we know DELETE operation deallocates the space and a new allocation can reuse that space which may still have data in it. When we do not zero out the existing space in case of instant file initialization there is a possibility that someone could read data that you thought had been deleted. This is a known security risk of this feature that must be considered while enabling it for your SQL Server Instance. Instant File Initialization is not enabled by default in SQL Server because of this security risk.

Enable Instant File Initialization for SQL Server?

Below are the steps you need to perform to enable Instant File Initialization.

Step 1- Login to server where you want to enable Instant File Initialization. This might be your database server or backup server.
Step 2- Open the Local Security Policy application. You can go to Control Panel->Administrative Tools->Local Security Policy. You can also open it directly by putting secpol.msc into run command prompt.

Step 3- In the left pane, expand Local Policies, and then click User Rights Assignment.

Step 4- In the right pane, double-click Perform volume maintenance tasks.

Step 5- Click Add User or Group and add any user accounts that are used to run SQL Server services. You can get SQL Server service account from SQL Server Configuration Manager or from services.msc console.

Now add account Domain\techyaz to above policy. It would be great if you create a group and add this account to that group and then add that group to given policy so that every time you don’t need to add user account here if SQL Server service account got changed.

Step 6- Once SQL Server service account will be added, it will look like below image. Click Apply, and then close all Local Security Policy dialog boxes.

Step 7- Restart SQL Server services to apply this change.

If you need to enable this feature on Cluster environment, make sure to perform this activity on all cluster nodes. If you are running with multiple SQL Server Instances, add all required security logins/group to this policy.

How to Check If instant file initialization is Enabled Or Not?

Now if you think how to know Instant File Initialization is enabled on your SQL Server Instance then here is the answer. You should check SQL Server logs to get the details about Instant File Initialization. When SQL Server service starts, database engine checks all details of SQL Server configurations and capture them into log file. You can see my error log file after restarting SQL Server service. Instant File Initialization is showing as enabled on this server.

2017-06-16 11:16:49.15 Server Database Instant File Initialization: enabled. For security and performance considerations see the topic ‘Database Instant File Initialization’ in SQL Server Books Online. This is an informational message only. No user action is required.

Advantages

Instant File Initialization improves performance of multiple database activities where large size of data file creation or Autogrow operation will perform. This can vastly speed up these operations as zero-initialization can take a long time for large files. Below are the operations that can take advantage from this feature.

• Data Load/Import
• New Database Creation
• Database Restore
• Autogrow Operation
• Data file size Increase
• Backup file Creation

Limitations

This feature will not work if:

• Your file is a log file.
• The account has not the required privilege.
• It is a sparse file – Database Snapshots.
• Transparent Data Encryption (TDE) is enabled.
• Trace flag 1806 is enabled to disable instant file initialization.

I hope you like this article. Please follow us on our facebook page and on Twitter handle to get latest updates.

Read More: 

• SQL Server Error 5009: One or more files listed in the statement could not be found or could not be initialized
• Fix SQL Server Msg 5144 and 5145 for long or failed autogrow of data and log files
• Manage Transaction Log File during Data Load

The post What is Instant File Initialization and How to enable it on SQL Server Instance? appeared first on Techyaz.com.