SQLvariations: SQL Server, a little PowerShell, maybe some Hyper-V Rotating Header Image

SQL Server

T-SQL Tuesday #31 – Logging

I’m excited to be hosting this month’s T-SQL Tuesday.  This month we’re talking about logging.  Logging comes in many form and fashions.  If you think about it, when you go to the grocery store with at big list, Do you put the items in the cart and then check them off the list?  If so, isn’t that a kind of write-ahead log?

I point that out because I don’t want anyone to constrain themselves to talking about logging within just SQL Server.  Please bring your ideas for file transfers, report generating, performance gathering, uptime monitoring and the like.  But don’t stop there!  This is an open invite to anyone that does anything in the SQL Server community.

If Karla Landrum ( blog | twitter ) wants to explain to us how on Earth she keeps track of all these SQL Saturdays around the world, that’s logging!  If Tim Radney ( blog | twitter ) wants to tell us how he makes sure he keeps in touch with all of the chapters he’s responsible for as a PASS Regional Mentor, that’s logging!

How you keep track of blog ideas, white papers you read, or however it is you life-hack *your* SQL world: please share it with us!

Rules
•Your post must be published between 00:00 GMT Tuesday June 12th, 2012, and 00:00 GMT Wednesday June 13th, 2012
•Your post must contain the T-SQL Tuesday logo from above and the image should link back to this blog post.
•Trackbacks should work, but if you don’t see one please link to your post in the comments section below so everyone can see your work

Optional, but highly encouraged…
•Include a reference to T-SQL Tuesday in the title of your post
•Tweet about your post using the hash tag #TSQL2sDay
•Consider hosting T-SQL Tuesday yourself. Adam Machanic keeps the list.

 

P.S.  Since I’m the host, any PowerShell topic on logging is cool too!

Resources for my SQL Server PowerShell Extensions webcast

imageI did another webcast for the Secrets of SQL Server webcast series today.  In the today’s webcast we built on some things I had demoed in a prior webcast and showed how PowerShell in conjunction with SQLPSX can really help you solve some common problems DBAs face.

I showed how you could pull size and free space information from all of your data files, then used the same technique to pull size information from all of your tables across every database on an instance.  Later we created a database and some tables, and then showed how to do some basic login comparisons, all with SQLPSX.

In addition, we stored the results of the frees pace and table size information in a database using the Out-DataTable and Write-DataTable functions.  Finally we wrapped up with a quick demo on how the new SQL Server cmdlets inside of the SQLPS Module in Denali combined with some new functionality in PowerShell v3 CTP1 can really make backing up a database with PowerShell a lot easier.

Here’s what you’ll need to recreate today’s demos;

I hope I see you next week at the PASS Summit or next month during another webcast, this time on customizing your own functions to work with the SMO.

New PowerShell Cmdlets in SQL Denali CTP3

CTP3 of SQL Denali is finally publicly available!

I woke up at 5 am this morning to kick off my download.  I’ve finished the install and I’m happy that I can finally report that we have new cmdlets for SQL Server. 

First off, after running Get-Module -ListAvailable I saw:

Manifest   SQLASCMDLETS
Manifest   SQLPS

This means that SQLPS inside of SQL Agent should finally allow you to import a module during your job step. 

There is now a Function called SQLServer that, from what I can tell, is the SQLServer Provider.  When under PS SQLSERVER:\> if you do a dir | Select pschildname you will find the following list:

PSChildName
———–
SQL
SQLPolicy
SQLRegistration
DataCollection
XEvent
Utility
DAC
IntegrationServices
SQLAS

From my notes it looks like XEvent, IntegrationServices, and SQLAS are the directories that have been added here.  Can’t wait to try out the IntegrationServices portion!  [Update]: Apparently I may need a defaul instance install for this to work.

I also noticed that Tab-Completion appears to be fixed!!  :-)   Also, SQL Registrations > ‘Database Engine Server Group’ are split up into instances like CMS was.  Not sure what that’s all about just yet.  In fact when I try to use it, it doesn’t seem to work.  Maybe I should read the help files.  But I think they may have just gotten CMS & RS reversed.  CMS isn’t working for me either yet.

When you drill down under XEvents and go to your Denali instaince you will see: Packages, & Sessions.  I guess this means that I finally need to read Jonathat’s blog series on X-Events.

When I imported the SQLPS module I received a warning about unapproved verbs.  Oh well, hopefully they’ll work that out soon.  Next I ran Get-Command -Module “SQLPS” and found the following list of cmdlets.

Name Module
Add-SqlAvailabilityDatabase SQLPS
Add-SqlAvailabilityGroupListenerStaticIp SQLPS
Backup-SqlDatabase SQLPS
Convert-UrnToPath SQLPS
Decode-SqlName SQLPS
Disable-SqlHADRService SQLPS
Enable-SqlHADRService SQLPS
Encode-SqlName SQLPS
Invoke-PolicyEvaluation SQLPS
Invoke-Sqlcmd SQLPS
Join-SqlAvailabilityGroup SQLPS
New-SqlAvailabilityGroup SQLPS
New-SqlAvailabilityGroupListener SQLPS
New-SqlAvailabilityReplica SQLPS
New-SqlHADREndpoint SQLPS
Remove-SqlAvailabilityDatabase SQLPS
Remove-SqlAvailabilityGroup SQLPS
Remove-SqlAvailabilityReplica SQLPS
Restore-SqlDatabase SQLPS
Resume-SqlAvailabilityDatabase SQLPS
Set-SqlAvailabilityGroup SQLPS
Set-SqlAvailabilityGroupListener SQLPS
Set-SqlAvailabilityReplica SQLPS
Set-SqlHADREndpoint SQLPS
SQLSERVER SQLPS
Suspend-SqlAvailabilityDatabase SQLPS
Switch-SqlAvailabilityGroup SQLPS
Test-SqlAvailabilityGroup SQLPS
Test-SqlAvailabilityReplica SQLPS
Test-SqlDatabaseReplicaState SQLPS

Analysis Services got some love too in the form of 11 cmdlets!  After inporting the SQLASCMDLETS module I ran this Get-Command -Module “SQLASCMDLETS” and found the following.

Name Module
Add-RoleMember SQLASCMDLETS
Backup-ASDatabase SQLASCMDLETS
Invoke-ASCmd SQLASCMDLETS
Invoke-ProcessCube SQLASCMDLETS
Invoke-ProcessDimension SQLASCMDLETS
Invoke-ProcessPartition SQLASCMDLETS
Merge-Partition SQLASCMDLETS
New-RestoreFolder SQLASCMDLETS
New-RestoreLocation SQLASCMDLETS
Remove-RoleMember SQLASCMDLETS
Restore-ASDatabase SQLASCMDLETS

More news to follow soon!

PowerShell Week at SQL University – Post 7

Deploying Code

Deploying SQL code with PowerShell can be very easy but why would you want to do that instead of just open a script in SQL Management Studio and run it?  The number 1 reason that I can think of is when you need to deploy to multiple destinations.  Let’s get started with something simple, we’ll deploy a table that I use in the post on collecting database sizes.

Invoke-Sqlcmd -ServerInstance Win7Netbook -Database CentralInfo -InputFile C:\temp\dbo.DatabaseFileSizes.SQL

That’s a pretty simple example and one that doesn’t offer too much any benefit over just opening up SSMS and running the script.   Let’s take it a little further with the loop construct and this time we’ll deploy Adam Machanic’s ( blog | twitter ) sp_WhoIsActive to a bunch of machines

$DeployFile = C:\temp\who_is_active_v11_00.sql
<# Loops through Registered SQL Severs and applys WhoIsActive #>
foreach ($RegisteredSQLs in dir -recurse SQLSERVER:\SQLRegistration\'Database Engine Server Group'\Development\ | where {$_.Mode -ne "d"} )
{
"Deploying to "+ $RegisteredSQLs.ServerName;
Invoke-Sqlcmd -InputFile $DeployFile -ServerInstance $RegisteredSQLs.ServerName -database master}

So now we can see how we can deploy a SQL Script to multiple machines with just 4 lines of PowerShell.  The script obviously doesn’t have to be deploying code.  It could inserting/updating data or even verifying permissions.

PowerShell Interfaces with Other Hammers

This month’s installment of T-SQL Tuesday is hosted by Pat Wright (blog | twitter). Pat says: “So the topic I have chosen for this month is Automation! It can be Automation with T-SQL or with PowerShell or a mix of both. Give us your best tips/tricks and ideas for making our lives easier through Automation.”

I have to tell you, this doesn’t get old:

imageI love that somebody on the other side of the planet is telling someone else that I ‘probably have a PowerShell script to solve their problem’. The funny thing is that a lot of the time I don’t have a script for the problem they’re trying to solve *yet*. I love it when someone gets referred to me and I don’t have a script because it gives me a chance to go explore the language and figure out something I don’t know. PowerShell makes this quest pretty fast.

imageIronically Paul White ( blog | twitter ) didn’t know I had a script for that, he was just making a joke. Well, actually, I didn’t have a script for that. I did however know right where to find the person that did; and that is even better than if I had a script of my own. Why is it better that I had to point someone some where else? Could I not have figured this script out own my own? No that’s not it at all.

imagePowerShell is NOT a hammer it’s an Automation Language. Better yet PowerShell is a Toolset, and it happens to come with some pretty freaking awesome hammers built in. We all understand that SQL Server isn’t just a hammer, that it’s an entire toolset. Inside of that toolset is: T-SQL, SQL Agent, Profiler, SSIS, the SMO, SSRS, SSAS etc… just to name a few.

The cool thing about PowerShell is it’s actually built to work with other hammers. So the reason that you hear me talking about PowerShell all the time is not because I’m replacing my T-SQL Hammer, I’m just automating it! Like this, here’s a quick script to figure out how long your SQL 2008+ instances have been running:

<# Number of Days since the Instance has been restarted #>
$InstanceList = "WIN7NetBook", "WIN7NetBook\R2", "WIN7NetBook\SQLExpress"
foreach($Instance in $InstanceList)
{
Invoke-Sqlcmd -Query "SELECT @@SERVERNAME AS 'ServerName'
, DATEDIFF(D, Sqlserver_start_time, SYSDATETIME()) AS 'NumberOfDays'
  FROM sys.dm_os_sys_info" -ServerInstance $Instance -Database master
}

The automation doesn’t stop with SQL Server. You can find plenty of posts form people in our SQL Community on working with things like Red-Gate’s SQL Compare, Outlook, Active Directory, and Subversion just to name a few. The key here is that all of these people are automating something adjacent to their SQL world and they’re using PowerShell to make it happen.

Heck just last week I wanted to know how many sessions had been submitted to SQL Saturday #67 in Chicago. Do you think I went to the schedule page, copied the list of sessions into Excel and then checked to see what line number the last session was on to get my count?? Heck NO!! Last year I wrote a script for Allen Kinsel ( blog | twitter ) to count how many people at the PASS Summit had registered their twitter handle when the signed up. I took that script, changed the link, changed the term I was looking for and presto! I now know that 105 sessions have been submitted to SQL Saturday #67.

That has nothing to do with SQL Server right?!… Except for that whole bit about it being a SQL Saturday that I was interested in.

<#Start up a web client and download the web page into a text file#>
$url="http://www.sqlsaturday.com/67/schedule.aspx"
$file="c:\temp\DownloadSession$(((Get-Date).ToString("yyyyMMddHHmm"))).txt"
$webclient = New-Object system.net.webclient
$webclient.DownloadFile($url,$file)            

<#Sift through the file for lines that include "viewsession" #>
(Select-String -path $file -Pattern "viewsession").count

OK, so what’s your point?

My point is that I wouldn’t have even had the twitter script that I started with if Allen Kinsel hadn’t taken a chance and sent me an email at noon the Friday before the PASS Summit and asked me if I could use PowerShell to count the number of attendees with twitter handles.

Again, what’s your point?

My point is Use PowerShell to make one tool talk to another.

And…

And tell people faced with an annoying problem that you think I’ve got a script for that because whose knows, maybe I do!

Why I Need a Week of SQLskills Immersion

SQLskills is giving back to the community with a free seat to their training class in Dallas next month (Feb. 21st – 25th)

To answer Paul Randal’s question of why I should get to win a week of training from SQLskills this is what I have to say:

Plain and Simple.  I’m a Second Generation DBA with over a decade of experience with SQL Server.  I want some deep level ideas for some new PowerShell scripts to simply tasks for DBAs everywhere.  I’ll also spend countless hours telling all my fellow DBAs here in Atlanta what all I learned until they can take it no more and demand this level of top-notch training for themselves!  Open-mouthed smile

I’ll get started with the PowerShell scripts right now even.  This little script will search twitter for a giver hashtag, in this case: #SQLskills.  It will then save everything it finds off to an XML file and then query it:

$SearchHashtag = "sqlskills"            
            
$wc = new-object net.webclient            
$url = "http://search.twitter.com/search.atom?q=%23$SearchHashtag"            
$OutFile = "c:\temp\Download_$($SearchHashtag)_$([DateTime]::Now.ToString('yyyyMMddHHmmss')).xml"            
$wc.DownloadString($url) | add-content $OutFile;            
            
[xml]$tweetxml = get-content $OutFile            
            
$tweetxml.feed.entry | Format-List            
            
($tweetxml.feed.entry).count            

PowerShell Week at SQL University – Post 5

In the previous posts we’ve just been poking around with PowerShell and trying to make the examples something that actually means something to a SQL person whenever we ca.  There are quite a few more language constructs that we need to cover but we have enough info to start recouping the time we’ve already invested.  Now it’s time to do one of those tasks that I just love to do with PowerShell.  We’re going to loop.  And we’re going to loop in a way that’s far easier than in any part of the SQL language.

We’re going to cover 3 different sources for our loop that are the most common for DBAs to use: table, text file, Registered Server/Central Management Server.  After that we’re going to do a double loop and then it’s time for you to find something to do with these.

Text File

In this chunk of code we’re just going to read from a simple text file from our local hard drive and then loop through the instances in that file one at a time.  You put one instance on each line of the text file.  Don’t put it in quotes unless you’re using a non-standard port number.  This is the easiest method because each “row” that comes out of the text file only has one property.  We’ll find out why that’s important in the next example.  In the meantime setup your text file and test it by running just this: Get-Content C:\PowerShell\AllInstances.txt.

foreach ($Instance in Get-Content C:\PowerShell\AllInstances.txt)
{
$Instance;
Invoke-Sqlcmd -ServerInstance $Instance -Database master -Query "SELECT  object_name ,
        counter_name ,
        instance_name ,
        cntr_value ,
        cntr_type
  FROM sys.dm_os_performance_counters"
}

 

Database Table/ Query

In this example we could be reading rows out of a table or running a more complex query to determine the list of databases that we want to run out query against.  This may actually seem easier to most database people and it is.  We’ve got a centrally located table and we can just look at it and know that we can change the first query to select a list of databases from somewhere else.  But there’s a really important thing to know if you swap out the query.   This approach is sending one usable property (column) to the foreach loop and it’s called “name”.  If you change the query and the column ends up being called database_name, you’re going to have to change $($db.name) to be called $($db.database_name).  Otherwise you’re going to loose a lot of hair and get really ticked after about 20 minutes like I did!

foreach ($db in invoke-sqlcmd -query "SELECT name  FROM sys.databases WHERE owner_sid !=0x01" -database master -serverinstance Win7NetBook )
{$db.name;
Invoke-sqlcmd -Query 'SELECT *
  FROM sys.dm_exec_procedure_stats' -ServerInstance  Win7NetBook -Database $($db.name)
}

 

Registered Servers/ Central Management Server

Before you get started looping through your Registered Servers you’ll need to run this: Import-Module Agent if you want to do this exact example.  What we’re looking for here is all the jobs that have failed in the last 3 days in our “QA” group of servers.  This example should be easy enough for everyone to tweak on their own.  If you get stuck just remember to do Get-Help -Full Set-AgentJobHistoryFilter
(Huge thanks to Chad Miller for helping me put together this demo so that it would be a fast one to run!)

$filter = Set-AgentJobHistoryFilter -startDate $(get-date).AddDays(-3) -endDate $(get-date) -outcome 'Failed'

foreach ($RegisteredSQLs in dir -recurse SQLSERVER:\SQLRegistration\'Database Engine Server Group'\QA\ | where {$_.Mode -ne "d"} )
{
Get-AgentJobHistory $RegisteredSQLs.ServerName $filter | where-object {$_.StepID -ne 0}
}

 

Double Loop

This has to be one of my favorite PowerShell scripts of all time (so far).  I had to run a query against every database in a group of over 10 servers.  I’ve changed this one around a little but I’m sure you’ll find a use for it!  (Think permissions.)

foreach ($RegisteredSQLs in dir -recurse SQLSERVER:\SQLRegistration\'Database Engine Server Group'\QA\ | where {$_.Mode -ne "d"} )
{
    foreach ($DBName in invoke-sqlcmd -query "SELECT name
  FROM sys.databases WHERE name in ('AdventureWorks',
            'AdventureWorks2008',
            'AdventureWorks2008R2',
            'AdventureWorksDW'
            ) " -database master -serverinstance $RegisteredSQLs.ServerName )
            {
                    invoke-sqlcmd -query 'SELECT *
        FROM sys.dm_db_index_usage_stats' -ServerInstance $RegisteredSQLs.ServerName -database $DBName.name
            } #EndOfTheFoundDatabasesLoop
} #EndOfTheRegisteredServerLoop

 

Homework

Try out each of these methods and think up something you could use this for.  If you hit on something that save you some clicking around in SQL Management Studio please mention it in the comments.

PowerShell Week at SQL University – Post 4

Recap

So far we’ve walked though how to turn on PowerShell and add modules, adding SQL Server 2008 snapins and using PowerShell variables, what cmdlets and functions are, what providers are and how to work with methods & properties.

Profiles

PowerShellProfilesToday we focus on load things in our profile so that they are always available in PowerShell.  First, some basics:

What is a Profile?  A profile is nothing more than a PowerShell script that gets run as soon as you open up a given PowerShell window.

Why would we want to do this?  I use PowerShell almost exclusively for automating work with my SQL Servers.  When I’m ready to do something I don’t want to start by reloading the same toolset every time I open it. I want to click the icon and be ready to go.  The profile saves me from this repetitive loading.  I update my profile as I find cool new scripts that I want to add.

You are not limited to using a single profile in PowerShell.  Personally, I use one of three profiles depending on the task at hand.  Here is how that works. I use the PowerShell ISE or other third party products as my primary way of writing PowerShell scripts (.ps1) and rarely use the “console.”   I only use the console if I need to troubleshoot something that might have to do with STA/MTA. 

When I open up the PowerShell ISE it’s uses it’s own profile CurrentUserCurrentHost that is stored in C:\Users\USERNAME\Documents\WindowsPowerShell\Microsoft.PowerShellISE_profile.ps1 (picture-right).  Inside of that file on my machine it currently looks like this:

import-module ISEPack
cd C:\Users\Aaron\Documents\PoSh\Load
. ./invoke-sqlcmd2.ps1
. ./Write-DataTable.ps1
. ./Out-DataTable.ps1
. ./Add-SqlTable.ps1
. ./Export-CSV-Append.ps1
. ./invoke-sqlcommand.ps1
. C:\Users\Aaron\Documents\PoSh\Tasks\Get-DisksSpace.ps1
cd c:\temp\

Whatever is in this file only gets loaded into my session when I open the ISE.  Nothing above gets run when I open up the console.  These are items I use in ISE, but not in the console PowerShell window.

There are items I use in both ISE and the console.  These items live in a “super profile” of sorts since I want these items no matter how I access PowerShell CurrentUserAllHosts (which is located in C:\Users\USERNAME\Documents\WindowsPowerShell\profile.ps1). Profile.ps1 (picture-center) gets run every time I open up any PowerShell “host” including 3rd party tools.  This also includes the SQLPS host which may cause errors messages to appear when you open it or run a SQL Agent Job.  These error messages don’t impact a PowerShell step in a SQL Agent Job from running.

Here’s what I have inside of my Profile.ps1 file.

Add-PSSnapin SqlServerCmdletSnapin100
Add-PSSnapin SqlServerProviderSnapin100
Import-Module SQLServer

Whenever I open the ISE both the Microsoft.PowerShellISE_profile.ps1 and the Profile.ps1 scripts get run.

PowerShell.exe (the little blue “host”) has it’s very own CurrentUserCurrentHost profile that it loads.  That’s the exact same name as the profile that we talked about the ISE loading so does that mean that it’s loading that same file?  No.  Each one is configured to load it’s own CurrentUserCurrentHost from a different location.  So if there are items I want in console world only I have a place to store them thanks to C:\Users\USERNAME\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1.  Since I rarely use the console my profile there sat empty until recently.  I only added items after I spotted a post by Oisin Grehan ( blog | twitter ) on Implementing a Matrix-Style Console Screen Saver and I HAD to try it.  This only works in the console and not in the ISE so it made my console profile the perfect place for implementing this.  Now my PowerShell_profile.ps1 file (picture-left) looks like this:

Import-Module screensaver
set-screensavertimeout (new-timespan -minutes 5)
enable-screensaver
Start-ScreenSaver

To edit one of the profiles that only load for the console or the ISE all you have to do is open them up and run notepad $profile.  It will figure out which file is the right one for that environment and open it for you.  To edit the profile that is run in every PowerShell environment just run notepad $profile.CurrentUserAllHosts.  If your WindowsPowerShell directory isn’t there it will not automatically create that for you and you will get an error message stating that it “can’t find the specified path“.  I’ve written a little script to check for that and create one for you if it’s not already there:

$myPowerShell = $HOME
$myPowerShell +="\Documents\WindowsPowerShell"
if ((Test-Path $myPowerShell) -eq "true") {"$myPowerShell is already there mate"}
else { "you need to make one"
mkdir $myPowerShell  }

Homework Assignment:

Setup one of your profiles to at least have one of the modules from the SQLPSX project and the SQL Server Snapins loaded up every time you start up the PowerShell ISE. Which profile to use is completely up to you.  Feel free to play around with your choices.  Optionally you can also include the Get-DisksSpace function like I do; but you don’t have to.

Let me know how this goes for you by dropping a note it the comments.  In tomorrow’s post we are going to work with multiple SQL Servers so you’ll need these things up and running for that.

If you’re confused by where your files are you can just run this script that I lifted off of the HeyScriptingGuys site to have it tell you where all of the profiles for your session have been loaded from.  Keep in mind that we’re not touching the ones in the Windows directory.

$profile |

Get-Member -MemberType note* |

Format-Table -Wrap -AutoSize -Property name,

@{ Label=“Path”; Expression={$_.definition -replace “System.String”,“”} }

PowerShellProfiles

In Case you would like more to play with…

There is a lot more that can be done with PowerShell Profiles but these are the basics you need to get up and running. If you want to dive deeper check out these posts over on the Hey Scripting Guy Blog.  Go here to download the SQLServer or ISEPack modules; they are both part of larger projects.  If you’d like some of the scripts that I reference in my profile you can grab the Get-DisksSpace one here and I’ll be blogging about the rest later this week.

Be Advised:

Everything that you put into you profile eats up RAM.  Thankfully most scripts are only a few kilobytes.  Keep in mind that the more items you add into your profile the longer that it is going to take to load.

imageIn case you’re wondering if that “Matrix Style Console Screen Saver” is worth having around, here is what it looks like after a few minutes:

That’s all that I’ve got for you today, see you back tomorrow!