Tuesday, May 1, 2012

Already? Sheesh, that was quick.


Great, just as I was getting the hang of SQL 2008R2 (still no luck getting the hang of Thursday), 2012 has come out (with CU1 being released just a few weeks ago). I was one of the many whom waited with bated breath for the virtual launch event and watched in awe of the fail. I am excited for the new features, especially the Always On, Availability Groups. With 2012 database mirroring is going the way of the stage coach. Which puts a huge crimp in my HA/DR plans. Currently I am utilizing mirroring with TDE, which I have yet to find any documentation on how to use TDE with AG. Granted I looked days after release, so I am hoping that there is more out there on the intertubes now. I am also looking at the newest interface for “power users” for data modeling and the SharePoint/SSRS integration.

We upgraded to 2008R2 almost a year ago now, what I have to decide is if it is worth the upgrade to 2012 so soon. I will have to talk with our developer(s) and have them check out any of new developer tools or enhancements and see if they think it is worth it. I would like to stay fairly current with our database infrastructure. I have to worry about the changes to the licensing model.

I also have to make sure that our various internal programs and 3rd party software will be compatible with the newest version. With the current shift that is going on in house, I have to at least make sure that MS Dynamics CRM 2011 works fully with SQL 2012. I am sure it will, but with Dynamics soon to be the center of business, I cannot afford to go with “I’m sure everything will work”.

Tuesday, February 14, 2012

On the shoulders of giants

In my previous post I had said that I was going to break out my maintenance plans and roll my own scripts.
That was a dumb idea.
Sure, writing something from scratch would've given me knowledge, yadda yadda yadda, and experience. But there is also the tested and true, "don't reinvent the wheel". While wanting to the the former, I subscribed to the latter. Off to The Google, I went. The first hit was Ola Hallengren's website.

When I was perusing the site, I kept thinking "why didn't I discover this sooner?"

After setting up the nuts and bolts of the maintenance solution on my local instance, off I went to check things out. The solution out of the box, sets up agent jobs without schedules. The various jobs are full backups of the system databases, user databases, diff and log backups of the user databases. There is also a job that cleans up the solution's own logging table. That table will get full quickly as it will log every index change, stat update, log/diff/full backup, every DBCC that is run etc... It will also create DBCC jobs for system and user databases. Penultimately it will create an agent job that will run to "optimize" indexes.

For the backups, we have a different LSA for each of our user databases, so I couldn't just use the same job for all of them. I just copy and pasted the contents of the job steps in each of the log, diff, and full jobs (that were created automatically) into new agent jobs that I had created. The best part is that you can specify which databases that you don't want to be backed up. 
 
The DBCC and Index Optimize stored procs take the same format in regards to which databases that you want to do the work on. For our environment I have added DBCC and Index Optimize as steps in each agent job along with the desired type of backup step.

I am glad that I have gotten away from maintenance plans. It feels good to try and swim in the deep end. I know I won't drown as the shoulders of giants aren't too far below me.

Sunday, January 29, 2012

I used to like Maintenance Plans, but not any more

When I started to hold the mantle of DBA I got jazzed about maintenance plans. They were easy, they made sense, and it was graphical. Three things I look for in any administration task I do. So I am thinking "awesome, I've got everything set, I'm backing up, doing maintenance on indexes and stats alike".

Yeah, no one likes maintenance plans.

Brent Ozar (web | Twitter)  explains why actual DBAs don't like maintenance plans. Who knew? People who've been at this longer than I have. I have come across many an odd error due to a failed maintenance plan. The best one yet has been an error about invalid credentials when connecting to a file share where the backup is being written (which is another post to debate whether or not this is a valid/accepted thing to do). The way I used to fix it? Change from using the DNS(\\archive\SQL\backup\) name to using the IP (\\192.168.1.107\SQL\backup\) address. No, I'm not kidding. Brent (and probably others) is correct in saying that maintenance plans just do weird things.

I have decided that enough is enough. I have the self confidence now with my understanding of the way things work that I am going to take the plunge and convert my maintenance plans. I have broken down my maintenance plans into pieces. Instead of doing DBCC, rebuild/reorg indexes, update stats, and backing up all in one plan, this will be done all in one job. Each plan task will be broken out into t-sql scripts and made into steps into the main agent job.

In following postings, I will go step by step of the deconstruction of my maintenance plans.


Sunday, January 8, 2012

SQL Family

Well I might as well throw my unknown hat into the ring. There was/is/has been a buzz around the Microsoft SQL Server community about the idea of "SQL Family". Well I am the first one to say that I am anything but active in the community, I have been known to call on family members for help. I am not even sure if I can say that I am a part of this family, part of the community, sure.

Like most lurkers I have the sqlhelp hash tag as a column in my TweetDeck. I try to offer advice where I am almost fairly certain I won't stick my foot in my mouth (which is few and far between the offering of help, not the sticking of feet in mouth)). This is somewhat a tragedy of the commons, as a few people are always there to offer help where need to the majority. I do feel a bit bad about abusing the help being offered.

I'm not even sure I belong, since being a DBA is only a portion of my job description. The main part consist of being a systems administrator. Do I still count for being a part of "the family"? I like being a sys admin, I like being a DBA I know at one point soon I'll have to choose between the two, as a person can only do one thing well, or multiple things in a mediocre manner.

I'd like to get more involved in the SQL community, however my wife and I just had our first son little under 4 months ago. Free time is something I only have at work...I kid, I am always busy at work. Time and monetary resources are a bit short at the moment. The most involvement that I can muster is to troll the dba.stackexchange board, and that's not even MSSQL centric. 


So for now, I'll sit on the side lines and every now and then add my two cents.

Saturday, December 31, 2011

Decimal Showdown

In setting up replication between a MSSQL publisher and a MYSQL subscriber, I have come across an annoyingly maddening issue. If there is a published column that has the decimal data type, it will cause replication to fail with an obscenely obtuse error. "Multi-step OLEDB error". Yeah try searching for that.
Did you find any reference to replication between MSSQL and MYSQL?
No?
Did you find posts about errors about data types?
Which led me to my next thought.

It came down to me doing a trial and error method going column by column changing the datatype on the MYSQL side, while leaving the MSSSQL datatype untouched. My method was, as follows:

  1. Stop replication
  2. Change data type of MYSQL column
  3. Delete all data from MYSQL table
  4. Reinitialize subscription
  5. Cross fingers

This looped for either 5 or 6 times, I have chosen to forget. Off I went searching The Google for any sort of help. I came up with a promising LiveJournal blog post. While helpful for background information of the 2006 difference between the two decimal data types in question, it's from 2006. There has been a precision change in the MYSQL decimal since then. The Books On-Line is a little thin in terms of the details of the decimal data type.

What is a novice DBA to do? Well use a different data type for MYSQL of course!

Let me forewarn, the data that is being replicated goes through no changes once it has been shipped over to the MYSQL server. All of the work of mathematics is done on the MSSQL side. The MYSQL is merely for displaying the data, not manipulating it. As such, I have elected to use the float data type. To reiterate I am doing no calculations on the MYSQL side, this must be said because using float is a really bad idea.

Don't believe me? Just ask this commentator on a StackOverflow thread. Using float to do calculations will give you a number, however you can't be sure of the exactness of it. Sure, if you using whole numbers and not worrying about accounting values, you don't have to worry about rounding. I on the other hand, do. We have gone through extensive testing in preparation for the data type change.

Our replication setup is such that the tables on the MSSQL side are just copies of the tables that are used in production. The tables on the replication server are used in no calculations, in our testing there was no change in the values on either side of the fence.

So in conclusion, don't do what I have done, do not use the float type as an equivalent data type for replication between MSSQL and MYSQL. It only works in this one situation. I am still on the search for a  resolution to my problem. Perhaps a different numeric data type will swoop in and save the day? Perhaps some DBA much smarter than myself will tell me why can't the two decimal data types get along? Until then, I'll have to deal with using float in place of decimal. If anyone out there has a solution please for the love of all that is holy, email me.

Wednesday, July 20, 2011

Learning SQL the hard way


(Yes, I borrowed that format from Learn Python the Hard Way http://learnpythonthehardway.org/)
I am currently studying for the 70-432 (SQL 2008, Implementation and Maintenance). If you had asked me two years ago if I was going to be studying for a SQL certification that answer would be a large no. Yet here I am. Am I ready for this? Is it going to matter? I certainly hope.

I totally fell into being the DBA for the company I work for. I came into the company as a “help desk” type of work, all I needed to know about SQL was how to restart the services if it ever failed. There were no backups, no rebuilding of indexes, no updating statistics. There was only hoping it didn’t fail, and crossing of fingers that we didn’t need to restore data. 

I wish I could say I became a DBA with guns blazing, kicking slow-queries, and taking traces. I wish I could say I don’t have to worry about a mortgage. I must give a major thank you to the company, they sent me to classroom training for 3 days a week for 4 weeks about maintenance and implementation of SQL 2005. That opened my eye to what I didn’t know, which was a hell of a lot.  After the class was over, I came back to work charged with fresh ideas and ways to actually use what I learned. I had maintenance plans, I was running DBCC checks once a week, I was doing transaction logs every fifteen minutes, I was doing nightly backups, fulls on the weekends. There was an index, or a stat that I didn’t rebuild, reorg or update. I was a man on fire. I had a road map for the SQL infrastructure; I was reading blogs, forums anything I could get my hands on.
Then I had to take off my DBA hat because everything was running fine.

On went the Sys Admin hat. We had to make the switch from physical infrastructure to a virtual one, and that wasn’t going to switch itself. I have kept up with my DBA personality, we are upgrading to SQL 2008R2 for our environment. Which brings me back to studying for the MCTS 70-432 test, I had to learn on the fly for SQL 2005. As much as I liked the whole “trial-by-fire” thing, I’d much rather to try knowing what is coming at me.

Friday, April 8, 2011

Database Mirroring with Transparent Data Encryption and You

I was presented with a challenge at my place of work. We are in need of using TDE to cover ourselves and our data, we also needed to setup database mirroring to keep inline with our DR/BC model. So I was tasked with making these two things work with one another.

Now I have successfully gotten both technologies to work before, so the challenge was how to get them to work together. Below is what I have figured out.

This website served as the foundation of the TDE+Mirroring, but the author left out key steps that I had to scrounge to fill in.

Overview:
Setup TDE on Principle DB
Restore TDE keys/certs from Principle DB to Mirror DB
Setup Mirroring on Mirrored DB
Setup Mirroring on Principle DB


Below are the individual steps I have taken to setup TDE and Mirroring.

The steps are setup so that the t-SQL (in italics) is on top with an explanation underneath (in bold).

On Principle:
USE MASTER
GO
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'SomePassword';
go

Creating the “master key” for the SQL server that houses the principle database, make sure that the password is a strong one
-------------------------------------------------------------------------------------------------------------------

On Principle:
USE master;
OPEN MASTER KEY DECRYPTION BY PASSWORD = ‘SomePassword’;
BACKUP MASTER KEY TO FILE = ‘Location\MasterKeyName.key' ENCRYPTION BY PASSWORD = 'SomePassword';
GO

Right after creating the “master key” we issue the command to back that up to a safe location. Most commands that you issue on the principle database or master key, will have to be prefaced with opening the master key by the password.
-------------------------------------------------------------------------------------------------------------------

On Principle:
USE Master
CREATE CERTIFICATE CertName WITH SUBJECT = ‘CertFriendlyName’, EXPIRY_DATE = '3500-Jan-01';
Go

Here we are creating a certificate that will sign the keys, with no expiration date.


-------------------------------------------------------------------------------------------------------------------

On Principle:
BACKUP CERTIFICATE CertName TO FILE = ‘Location\CertName.cer'
          WITH PRIVATE KEY ( FILE = 'Location\CertKeyName.key', ENCRYPTION BY PASSWORD = 'SomePassword');
GO

Again, right after creating the cert we are backing it up. We also create yet again another key this time to encrypt the cert. Make sure that the key for the cert is NOT the same key as the “master key”, the code will fail and not with a helpful error.
-------------------------------------------------------------------------------------------------------------------

On Principle:
USE DatabaseName
CREATE DATABASE ENCRYPTION KEY
WITH ALGORITHM = AES_256 ENCRYPTION BY SERVER CERTIFICATE CertName;
GO

This step creates the database encryption key using the cert we created above. You can change the type of encryption; just check Books Online for options.
-------------------------------------------------------------------------------------------------------------------

On Principle:
USE master;
GO
ALTER DATABASE DatabaseName SET ENCRYPTION ON
GO

So finally we turn TDE on for the principle database.


We now turn our efforts to the server that will host the mirrored database.

On Mirror:
USE master
RESTORE MASTER KEY
    FROM FILE = 'Location\MasterKeyName.key'
    DECRYPTION BY PASSWORD = 'SomePassword'
    ENCRYPTION BY PASSWORD = 'SomePassword';
GO

We are restoring the backed-up master key file that we created on the principle server to the mirror server
-------------------------------------------------------------------------------------------------------------------

On Mirror:
USE Master;
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'SomePassword'
CREATE CERTIFICATE CertName   
FROM FILE = 'Location \CertName.cer' WITH PRIVATE KEY ( FILE = 'Location \CertKeyName.key', DECRYPTION BY PASSWORD = 'SomePassword');
GO


We are restoring the cert to the mirror, using the cert name and passwords that were created on the principle.
-------------------------------------------------------------------------------------------------------------------

On Mirror:
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'SomePassword'
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
GO

This is my favorite part. This little three line snippet took me 3 days to figure out. I found it in the comments of the website linked above. Here we are altering the master key with the service master key of the mirrored server.
-------------------------------------------------------------------------------------------------------------------

Now that we have all of the TDE stuff in place on both the principle and the mirror, we need to backup the principle db, and restore it, and then enable mirroring!

On Principle:
USE DatabaseName
GO
BACKUP DATABASE DatabaseName
TO DISK = 'Location\DatabaseName_FULL.bak'
          WITH COMPRESSION,
                    NAME = 'Full Backup of DatabaseName';
GO

--T-Log Backup of TDE
USE DatabaseName
GO
BACKUP LOG DatabaseName
TO DISK = 'Location\DatabaseName_LOG.trn'
          WITH COMPRESSION,
               NAME = 'Log backup of DatabaseName’
GO

The backing-up of the principle database is standard full backup and transaction log backup.
-------------------------------------------------------------------------------------------------------------------

On Mirror:
USE MASTER
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'SomePassword'
RESTORE DATABASE DatabaseName
   FROM disk = 'Location\DatabaseName_FULL.bak'
      WITH NORECOVERY,
       REPLACE,
       MOVE 'LogicalDatabaseName' TO 'Location\DatabaseName.mdf',
      REPLACE,
      MOVE 'LogicalDatabaseLogName' TO 'Location\DatabaseName.ldf'
CLOSE MASTER KEY
GO

So when performing the restore make sure that you open the master key, then close it when done. If you don’t know the “logical” names of the files, right-click on the database name, then select “files”, look at the first column for the “logical” names of the .mdf and .ldf files. Also make sure that you specify “WITH NO RECOVERY”, because we want to restore the transaction log after this.
-------------------------------------------------------------------------------------------------------------------


On Mirror:
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'SomePassword'
RESTORE LOG TDE
          FROM DISK = 'Location\DatabaseName_LOG.trn'
          WITH NORECOVERY;
CLOSE MASTER KEY
GO

For database mirroring to work, you must specify to restore the log backup “WITH NORECOVERY” the database has to be in a restoring state to initialize mirroring.
-------------------------------------------------------------------------------------------------------------------

Now on to the setting up of the database mirroring

On Principle/Mirror:
USE MASTER
SELECT * FROM sys.database_mirroring_endpoints
GO

This command will display the endpoints that are already on the server.
-------------------------------------------------------------------------------------------------------------------

On Principle/Mirror:
DROP ENDPOINT EndPointName

Issuing this command will drop the end point if need be.
-------------------------------------------------------------------------------------------------------------------

As a rule, I look for any existing endpoints to make sure that I am not duplicating any names.

On Principle/Mirror:
CREATE ENDPOINT EndPointName
          STATE = STARTED
          AS TCP ( LISTENER_PORT = PortNumber )
    FOR DATABASE_MIRRORING (
                   ROLE = PARTNER
                   );
GO

For my endpoint port number I use 7025 on the principle, and 7022 for the mirror.
-------------------------------------------------------------------------------------------------------------------
 
On Principle/Mirror:
SELECT dme.endpoint_id
,dme.name
,dme.principal_id
,dme.state_desc
,dme.role_desc
,dme.connection_auth_desc
,dme.certificate_id
,dme.encryption_algorithm_desc
,te.port
,te.ip_address
FROM sys.database_mirroring_endpoints dme
INNER JOIN sys.tcp_endpoints te
 ON dme.endpoint_id = te.endpoint_id AND te.type = 4
GO
This will check to make sure that the endpoints are started. This command was pulled from the comments section of the link posted above
-------------------------------------------------------------------------------------------------------------------

After both endpoints are created and started, it is time to issue the commands to start the mirroring.

On Mirror:
USE MASTER
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'SomePassword'
ALTER DATABASE Databasename
SET PARTNER = 'TCP://PrincipleServerName.Domain:PortNumber'
GO
CLOSE MASTER KEY
GO

As a rule, I always issue the “start” code on the mirror to connect to the principle first. The connection has to be the FQDN, eg, server.domain.local etc… I am not sure if the open/close master key is needed, but it doesn’t error out.
-------------------------------------------------------------------------------------------------------------------

On Principle:
USE MASTER
ALTER DATABASE DatabaseName
SET PARTNER = 'TCP:// MirrorServerName.Domain:PortNumber'
GO

This command will start database mirroring on the principle.
-------------------------------------------------------------------------------------------------------------------


So, after issuing this code on the principle server, you should have TDE and Mirroring all setup. To check, refresh the database list in SSMS, the databases should read “Principle, Synchronized” and “Mirror, Synchronized/Restoring”

You can right click on the principle database, and choosing tasks-> mirroring monitor to check it out!