Couple of days ago I installed SCVMM  in order to test it out, since we are developing our IAAS solution. Everything seemed great until I found out that SVCMM doesnt have any API which can be used to create/turn on/off  new machines.  Yes, I know everything can be scripted thru Powershell, but it sucks that we need to create separate applicatons that will give us that feature. Imagine a solution where PRTG, Nagios or any other tools monitors your server… sees that the CPU load went up ( or any other condition) and starts up a new virtual machine  using API?

Its funny how the guys at MS fails to see the basic requirements for a modern cloud solution; do I have to mention that after years of HyperV being RTM-ed, we still dont have an option to limit a machines uplink speed!? Come on guys…SERIOUSLY!?!?

I know that you already have this  little piece of software created, release it…pretty please!? I know that Microsoft sees HyperV in some giant corporation where they dont have a need to limit the uplink of a virtual machine, but IMHO, hosters are having a bigger “market” share  than big corporations, think about us, pretty please with sugar on top .

Oh, what about promiscous mode for network cards? You  have no idea how much would I like to use SNORT on my HyperV virtual switch, but…I cant

What about the limit on only 4 cores per virtual machine? This is a VERY limiting factor. I’m in a situation where i have a 2x six core, 48 GB ram server which I want to turn into ONE virtual machine ( live migrations, scaling, etc…), but I cant…I can only turn it into a 4 core, 44 GB machine, effectively, wasting other 8 cores!?

I would really love to get in touch with someone from Microsoft Virtualization team and send them a list of must-have changes Someone knows some insiders?

Let me make one thing clear, I love Microsoft, but lets face it, there is lots of work that needs to be done. Admit it or not, the guys at the “Linux Camp” have much more options and features regarding virtualization. I really, really love HyperV, but some limitations are just plain unlogical/stupid.

Sorry for a bitter post, but I had to throw out my frustrations

So…today I had a very interesting issue. All of a sudden Nagios went red with the message: CRITICAL : – total: 232.82 Gb – used: 212.67 Gb (91%) – free 20.15 Gb (9%) on one of our SQL server. I was like..”NO! It cannot be. I clearly remember that this machine had over 150 Gb of free space yesterday!”

I log in to the machine, and automatically go to the data folder of SQL server thinking that some customer has a HUGE! log file on his database; so, right click – Select All -> Properties… only 50 Gbs of data…hmm…not here!

At that moment I notice that the server isnt “jumpy” as usual, so I start Performance monitor and see that the Avg. Disk Queue is off the chart!  That definatelly isnt right…

Since that machine is using Windows 2003, theres no too good way to see what application is using so much disk I/O ( beside FileMon ) , I start the FileMon and se that the SQL service is writing HUGE amount of data to its ERRORLOG file.

By navigating to the folder, I see that the ERRORLOG file is over 120 Gb big so opening it in Notepad, Context or some other file was out of the question, since it would kill the machine completely.

So, lets turn to powershell once more. In order to be sure to not kill the machine, since its in production , I have moved the file to an empty machine to opened it there.

Powershell : cat errorlog +ENTER after that powershell started to list the entire content of the file, and it took about 15 mins until it reached the end. AMAZING

By examining the log, I notice ( it was hard to NOT notice it ) litterally millions of these entries:

Could not allocate space for object ‘dbo.tblComments’.'tblComments’ in database ‘DATABASEname’ because the ‘PRIMARY’ filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.

What happened here?

The site that was using that database is poorly written and was allowing everyone to post comments on the articles without any security check, so it soon reached the size limit set on the database.  By further digging on the issue, I saw that the bots are posting about 4-5 comments each second which comes to about 18000 comments/hour…multiply that by 15-16 hours it took to fill the error log to that size, we roughly come to 280000 comments…that’s A LOT of comments!

After disabling the site and sending the email to the customer, situation on the machine looks like this :

The big drop on the green line is the moment where I disabled the problematic web site.

Microsoft SQL Server 2005/2008 Express is a free edition of SQL Server that is ideal for learning, developing and powering desktop, web and small server applications, and for redistribution by ISVs.

Top Features

First, we need to have a query that will backup all databases to one folder… Well, here it is


DECLARE @name VARCHAR(50) -- database name
DECLARE @path VARCHAR(256) -- path for backup files
DECLARE @fileName VARCHAR(256) -- filename for backup
DECLARE @fileDate VARCHAR(20) -- used for file name
SET @path = 'E:\sqlbackups\' --path to the backup folder
SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112)
DECLARE db_cursor CURSOR FOR
SELECT name
FROM master.dbo.sysdatabases
WHERE name NOT IN ('tempdb')
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO @name
WHILE @@FETCH_STATUS = 0
BEGIN
SET @fileName = @path + @name + '_' + @fileDate + '.BAK'
BACKUP DATABASE @name TO DISK = @fileName
FETCH NEXT FROM db_cursor INTO @name
END
CLOSE db_cursor
DEALLOCATE db_cursor

This will backup everything except the “Temdb” database. So, lets take a look at the current state of our backup folder.

As we can see, only the backups from 21.11.2009 exist. So, what happens when we run the query using SQL Management Studio.

Here’s the query pasted in “New query window”.

Sooo…if everything went smoothly, we should get a message similar to this one :

Awesome! Now, lets take a look at our backup folder, are there any changes? Sure hope there is, because this post will totally suck if the backup didn’t complete

As we can see, this post will not suck after all Notice the new files, created on the 22.11.2009 . IT WORKS! All we have to do now is to automate the backup, so we don’t have to run the query everyday by hand.

We will do that using Task Scheduler (or Scheduled tasks in Windows 2003). We have to add a task to execute everyday in 3 AM.

The command we have to run is:

“C:\Program Files (x86)\Microsoft SQL Server\90\Tools\Binn\SQLCMD.EXE” -i “c:\pathtosqlquery\sqlbackup.sql”

Where “sqlbackup.sql” is the name of the file you have pasted the code mentioned before.

Here are a couple of pictures so you can see how to do it:

Now, click on “Create task”. That will click will start the task wizard. Enter the name of the task ( i.e “SQL Backups”.

After entering the name and choosing the option “Run wheneter user is logged on or not”, click on “Triggers”.

Click on “New”. It will bring up a new window

Choose the “Daily” option, and set the time when you want the task to run. I have set it to 3 AM .

ONE IMPORTANT THING!!! Dont forget to check the “Enable” checkbox , otherwise, you will end up scratching your head and cursing Microsoft and uncle Bill hehe.

After that, click on “Actions”, where we will define the program that should be runned… Click on “New”…

Paste this into the “Program/Script” box.

“C:\Program Files (x86)\Microsoft SQL Server\90\Tools\Binn\SQLCMD.EXE”

NOTE! – path to the “SQLCMD.exe” may be different on your system!!!

All that is left to do is to “Add arguments”

-i “c:\pathtosqlqueryfile\sqlbackup.sql”

Click on “OK”, enter the system password and thats it. Enjoy your daily backup.

HINT!!!! If you’re lazy like I am, you can use a powershell script to delete backup files older than X days. I have also covered that with a blog post. More info about that on this link.

Bye bye!

Surfing all over the net, I found a very handy powershell cheat sheet that was put together by Ben Pearce. Anyways, here’s the download link (just click on the picture).

Big hello to the guys from www.bug.hr

Tired of old point and click interface? Powershell is the new purple in computer world haha…Seriously!

So, here’s a little script that will add the Ip address you want to the existing firewall rule.

As you can see, it fairly simple, and there’s no need to explain it too much….


Function IP
{
param ($ipadress = $(Read-host "Enter IP address"))
$rule = "YOUR PREVIOUSLY CREATED RULE NAME"
Write-host "Adding address :" $ipadress -foregroundcolor "Red"
Start-sleep -s 5
netsh advfirewall firewall set rule name="$rule" new remoteip="$ipadress" action="allow"
}
IP


Only thing you have to do in the script is to enter your fireall rule name.

When you start the script, it  will ask you to enter desired ip address, and press enter. After that, wait 5 seconds and thats it

Cheers!

Recently I’ve found myself in a situation where I had to periodically delete files older than 7 days. I could have done it manually, but being busy, I often forgot to do it. The script I’m about to show you is a bit modified than the original, but, the principle is the same. Here I’ve added a posibilty to manually enter the directory name. In your script, you can define it, so it is being run automatically, without your intervention.

Here’s the code :

Function GetOldFile
{
param ($Dir = $(Read-host "Unesi putanju do direktorija")),
($Days = $(Read-Host "Brišem fileove starije od koliko dana?"))
$TargetFolder = $Dir
if (Test-Path $TargetFolder)

{
Write-host "DIREKTORIJ JE :" $TargetFolder -foregroundcolor "Red"
#Write-Host `a `a `a `a `a - odkomentirati ako zelimo da pc speaker vristi
Write-Host "UPS, krivi folder! Stisni 'Ctrl + C' za prekid - imas 5 sekundi za prekid" -foregroundcolor "Yellow"
Start-sleep -s 5
$Now = Get-Date
$LastWrite = $Now.AddDays(-$days)
$Files = get-childitem $TargetFolder -include *.log -recurse |Where {$_.LastWriteTime -le "$LastWrite"}
foreach ($File in $Files)
{write-host "Brišem file $File" -foregroundcolor "Red"; Remove-Item $File | out-null}
}
Else
{Write-Host "Direktorij $TargetFolder ne postoji! Provjeri path!"}
}
GetOldFile

The questions in the script are in Croatian, but anyone with a little bit of programming experience will be able to change it, because the script is pretty straight forward. But, if you’re lazy as most of us are, here is the translated script

Function GetOldFile
{
param ($Dir = $(Read-host "Enter destination folder path")),
($Days = $(Read-Host "Delete files older than how many days ?"))
$TargetFolder = $Dir
if (Test-Path $TargetFolder)
{
Write-host "Target folder path is :" $TargetFolder -foregroundcolor "Red"
#Write-Host `a `a `a `a `a - uncomment if you would like to hear the pc speaker #warning
Write-Host "WRONG FOLDER! Press 'Ctrl + C' to abort"-foregroundcolor "Yellow"
Start-sleep -s 5
$Now = Get-Date
$LastWrite = $Now.AddDays(-$days)
$Files = get-childitem $TargetFolder -include *.log -recurse |Where {$_.LastWriteTime -le "$LastWrite"}
foreach ($File in $Files)
{write-host "Deleting file $File" -foregroundcolor "Red"; Remove-Item $File | out-null}
}
Else
{Write-Host "Folder $TargetFolder doesnt exist! Recheck the folder path!"}
}
GetOldFile


Be careful with this script, because it will delete all the files in the subfolder also! Notice the “-recurse” in the script..

Now, to make your life even easier, you can add this script to scheduled tasks and set it to run whenever you need it.

First thing that we need to do is to set the folder path as a constant value. Instead of

param ($Dir = $(Read-host "Enter destination folder path")),
($Days = $(Read-Host "Delete files older than how many days ?"))

we need to define $Dir and %Days in the script. That is done like this:

$Days = "2"
$TargetFolder = "C:\pathtofolder"


So the script looks like this:

Function GetOldFile
{
$Days = "2"
$TargetFolder = "C:\pathtofolder"
if (Test-Path $TargetFolder)
{
$Now = Get-Date
$LastWrite = $Now.AddDays(-$days)
$Files = get-childitem $TargetFolder -include *.log -recurse |Where {$_.LastWriteTime -le "$LastWrite"}
foreach ($File in $Files)
{write-host "Deleting file $File" -foregroundcolor "Red"; Remove-Item $File | out-null}
}
Else
{Write-Host "Folder $TargetFolder doesnt exist! Recheck the folder path!"}
}
GetOldFile


Notice that I have removed couple of lines from that script that are not neccesary to the script to function properly. If you like, you can also remove the line where the script writes out the names of the files that are being deleted.

Second thing you need to do is make sure that Powershell is set to execute Powershell scripts, instead of only allowing interactive commands to be run in the Powershell environment.

Type the following at the Powershell command prompt:

set-executionpolicy RemoteSigned

After that, create a .bat file and name it whatever you like ( i.e dltfilesolderthanXYday.bat) and paste the following entry in:

powershell -command "& 'SCRIPTNAME.ps1' "

All that is left to do is to create a Scheduled Task in Windows. Add a task that will run the .bat file every number of days you need.

Enjoy!

Working as a Windows server administrator many times I found myself in a situation where I have to check 50+ Mb log file for only one e-mail address or some other term. Ofcourse, many of you are going to think: “Yeah, like that’s a problem, just press CRTL+F and type in the term ” ,and you’re right. That way rocks when you have 10, or maximum 20 entries where that term is mentioned ( i.e some e-mail address) in the entire log file .  But, what to do when you’re in a situation that a client wants to see the logs for his entire domain. All sent e-mail, all incoming e-mail, everything…then, it gets a little too much to do “crtl+f”  and copy/paste the result in some other file which you are going to send to the client.  Linux users have their way to do that, simply by : cat filename.txt | grep term > outputfile.txt .  By starting to explore powershell, my first task was to find a similar command to do the same thing on a Windows server. Doing a little searching on the web I found the solution that is really simple and similar to linux command hooray for me

The Unix “cat” command is used to concatenate and display files.   Given a file or filename list, it will print the contents of that file to standard output.  There are several options in the Unix command that are implemented with the following PowerShell arguments:

But, as we can see, no  “grep” command like on linux . Again, some more searching and I found a command called “findstr” .

Example:

c:\cat logs.txt  – will print all the content of that file.

#Software: Microsoft Internet Information Services 6.0
#Version: 1.0
#Date: 2009-09-28 15:30:08
#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken
2009-09-28 15:30:08 W3SVC1958603616 WIN7 89.201.164.205 GET /index.php - 80 - 89.201.144.106 HTTP/1.1 Mozilla/5.0+(X11;+U;+Linux+x86_64;+en-US;+rv:1.9.0.14)+Gecko/2009090217+Ubuntu/9.04+(jaunty)+Firefox/3.0.14 - - www.winblogs.net.win7.mojsite.com 500 0 0 1900 406 31
2009-09-28 15:30:08 W3SVC1958603616 WIN7 89.201.164.205 GET /wp-admin/css/install.css - 80 - 89.201.144.106 HTTP/1.1 Mozilla/5.0+(X11;+U;+Linux+x86_64;+en-US;+rv:1.9.0.14)+Gecko/2009090217+Ubuntu/9.04+(jaunty)+Firefox/3.0.14 - http://www.winblogs.net.win7.mojsite.com/ www.winblogs.net.win7.mojsite.com 200 0 0 2802 437 15
2009-09-28 15:30:08 W3SVC1958603616 WIN7 89.201.164.205 GET /wp-admin/images/white-grad.png - 80 - 89.201.144.106 HTTP/1.1 Mozilla/5.0+(X11;+U;+Linux+x86_64;+en-US;+rv:1.9.0.14)+Gecko/2009090217+Ubuntu/9.04+(jaunty)+Firefox/3.0.14 - http://www.winblogs.net.win7.mojsite.com/wp-admin/css/install.css www.winblogs.net.win7.mojsite.com 200 0 0 460 482 15

Now, lets say that we want to see the lines only where the word “username” is mentioned. As you can see, there is much other stuff in the output that we dont need, so we will simple do this:

c:\cat logs.txt | findstr username

and it will print out the lines where the term “username” is mentioned.

#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken
#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host


AWESOME, we got 60 percent of the work done! All we need to do now is to copy that output to a file. Yes, we could do copy/paste, but that is boring . All we have to do is add  ” > filename.txt” to the end of the cat command.

Example:

c:\ cat logs.txt  | findstr username > outputfile.txt

And bingo! All the lines where the term “username” is mentioned are now written to the file called “outputfile.txt” – ofcourse, you can name it whatever you like

It made my life easier Cheers!