Couple of days ago I installed SCVMM  in order to test it out, since we are developing our IAAS solution. Everything seemed great until I found out that SVCMM doesnt have any API which can be used to create/turn on/off  new machines.  Yes, I know everything can be scripted thru Powershell, but it sucks that we need to create separate applicatons that will give us that feature. Imagine a solution where PRTG, Nagios or any other tools monitors your server… sees that the CPU load went up ( or any other condition) and starts up a new virtual machine  using API?

Its funny how the guys at MS fails to see the basic requirements for a modern cloud solution; do I have to mention that after years of HyperV being RTM-ed, we still dont have an option to limit a machines uplink speed!? Come on guys…SERIOUSLY!?!?

I know that you already have this  little piece of software created, release it…pretty please!? I know that Microsoft sees HyperV in some giant corporation where they dont have a need to limit the uplink of a virtual machine, but IMHO, hosters are having a bigger “market” share  than big corporations, think about us, pretty please with sugar on top .

Oh, what about promiscous mode for network cards? You  have no idea how much would I like to use SNORT on my HyperV virtual switch, but…I cant

What about the limit on only 4 cores per virtual machine? This is a VERY limiting factor. I’m in a situation where i have a 2x six core, 48 GB ram server which I want to turn into ONE virtual machine ( live migrations, scaling, etc…), but I cant…I can only turn it into a 4 core, 44 GB machine, effectively, wasting other 8 cores!?

I would really love to get in touch with someone from Microsoft Virtualization team and send them a list of must-have changes Someone knows some insiders?

Let me make one thing clear, I love Microsoft, but lets face it, there is lots of work that needs to be done. Admit it or not, the guys at the “Linux Camp” have much more options and features regarding virtualization. I really, really love HyperV, but some limitations are just plain unlogical/stupid.

Sorry for a bitter post, but I had to throw out my frustrations

Here you can download the files for fixing mysql issues ( deleting database ) on WSP 1.0.1.0, 1.0.2 , 1.1.0 and 1.1.2

INSTRUCTIONS :

Replace “websitepanel.providers.database.mysql.dll” file in “server\bin” folder with the one you downloaded. These fixes fixed my problem on all servers EXCEPT the one running mysql 5.0.27 version….ones runing 5.1 and 5.0.77 works fine. I Will update if I get this solved too…

Currently I am getting this error on 5.0.27 server: ERROR: ‘MySQL Server 5.0′ DeleteDatabase System.Data.EvaluateException: Cannot perform ‘=’ operation on System.Byte[] and System.String.

FIX for 1.0.1.0 version can be found HERE

FIX for the 1.0.2 version can be found HERE

FIX for the 1.1.0 version can be found HERE

FIX for the 1.1.2 version can be found HERE – UPDATED!!!

Hope it will be helpfull.

If you have any questions, just let me know.

Ofcourse, these fixes are provided AS-IS . I dont guarantee it will work on your machine, and I cannot take responsibility for any damages cause by it.

So…today I had a very interesting issue. All of a sudden Nagios went red with the message: CRITICAL : – total: 232.82 Gb – used: 212.67 Gb (91%) – free 20.15 Gb (9%) on one of our SQL server. I was like..”NO! It cannot be. I clearly remember that this machine had over 150 Gb of free space yesterday!”

I log in to the machine, and automatically go to the data folder of SQL server thinking that some customer has a HUGE! log file on his database; so, right click – Select All -> Properties… only 50 Gbs of data…hmm…not here!

At that moment I notice that the server isnt “jumpy” as usual, so I start Performance monitor and see that the Avg. Disk Queue is off the chart!  That definatelly isnt right…

Since that machine is using Windows 2003, theres no too good way to see what application is using so much disk I/O ( beside FileMon ) , I start the FileMon and se that the SQL service is writing HUGE amount of data to its ERRORLOG file.

By navigating to the folder, I see that the ERRORLOG file is over 120 Gb big so opening it in Notepad, Context or some other file was out of the question, since it would kill the machine completely.

So, lets turn to powershell once more. In order to be sure to not kill the machine, since its in production , I have moved the file to an empty machine to opened it there.

Powershell : cat errorlog +ENTER after that powershell started to list the entire content of the file, and it took about 15 mins until it reached the end. AMAZING

By examining the log, I notice ( it was hard to NOT notice it ) litterally millions of these entries:

Could not allocate space for object ‘dbo.tblComments’.'tblComments’ in database ‘DATABASEname’ because the ‘PRIMARY’ filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.

What happened here?

The site that was using that database is poorly written and was allowing everyone to post comments on the articles without any security check, so it soon reached the size limit set on the database.  By further digging on the issue, I saw that the bots are posting about 4-5 comments each second which comes to about 18000 comments/hour…multiply that by 15-16 hours it took to fill the error log to that size, we roughly come to 280000 comments…that’s A LOT of comments!

After disabling the site and sending the email to the customer, situation on the machine looks like this :

The big drop on the green line is the moment where I disabled the problematic web site.

Fact number one: If you have a streaming server, you must install this extension!

Great, now that we’ve got this fact covered, we can move on.

IIS site says:

Bit Rate Throttling, an IIS Media Services extension, saves money on network costs by metering the download speed of media and data files. For media, Bit Rate Throttling accomplishes this by automatically detecting the encoded bit rates of 11 common media formats, such as Windows Media Video (WMV), Flash Video (FLV), and MPEG 4 (MP4), and then throttling the response to the client. For any other file or MIME type, Bit Rate Throttling allows administrators to configure custom throttling rules.

Now, lets explain the same thing in a bit simpler way.

If you have a server which streams any kind of media files, Im sure you have noticed extremely high bandwidht usage on that machine. In my experience, over 40% of the bandwidth is wasted! WAIT! WHY, HOW?

Answer is simple!

Scenario 1 ( server without Bit Rate Throttling extension ):

Lets say that you have a 100 Mbit link on your machine. A visitor with a 20 Mbit link will come to your site and click “play” on the newest promo movie you have uploaded.  What will happen, the server will send the ENTIRE video to the visitor in couple of seconds ( buffering ) and on the half of the movie, the visitor will see that he doesnt like your movie and go away.

What happened here? Half of the downloaded movie is wasted. The visitor downloaded the entire movie, but only looked at half of it. Efectively, you’ve wasted 50% of that movies bandwidth ( in a 100 Mb file, thats 50 Mb of bandwidth wasted )

This is where Bit Rate Throttling jumps in and saves the day!

Scenario2 ( Server with Bit Rate Throttling Extension installed ) :

Lets say that you have a 100 Mbit link on your machine. A visitor with a 20 Mbit link will come to your site and click “play” on the newest promo movie you have uploaded.  What will happen, the server will see that the movie is encoded at 700 kbps ( example ) and reduce the speed its sending you the movie ( buffering )  to 800 kbps or 850 kbps ( depening on what setting you choose ) so that you still have a smooth playback and some buffer in reserve ).   In this case, amount of bandwidht that could potentially be wasted is a lot smaller than the amount of bandwidth that would be wasted without this extension.

As I said before, improvements in bandwidth costs can be improved up to 40%! Thats not a small amount!

Features of Bit Rate Throttling:

• Seamless integration into IIS 7.x Manager
• Out-of-the box compatibility for the following media types: .asf, .avi, .flv, .m4v, .mov, .mp3, .mp4, .rm, .rmvb, .wma and .wmv
• Dynamic throttling adjustment based on:
  • The file types and their encoded bit rate
  • The available bandwidth of the connection
  • Default and maximum throttling rates
  • Concurrent connection limits
  • Target throughput rate for the site
• Throttling rules configurable at various levels including server, site and virtual directory
• Fast Start experience for optimal user experience
• Extensibility to enable other media formats and file types
• Better-together solution when combined with Web Playlists

Have a nice day! Alesandro

In the beginning, some c/p from IIS site:

IIS Application Warm-Up for IIS 7.5 enables IT Professionals to improve the responsiveness of their Web sites by loading the Web applications before the first request arrives. By proactively loading and initializing all the dependencies such as database connections, compilation of ASP.NET code, and loading of modules, IT Professionals can ensure their Web sites are responsive at all times even if their Web sites use a custom request pipeline or if the Application Pool is recycled.

Now, ME!

At first glimse you can say : “WOW” this extension rocks!!! I dont have to wait for my application to compile! My site will load much much faster!

And, imagine, you would be completely RIGHT! This is a great extension, but ( yeah, there’s a but ), I would NEVER install this extension on a shared hosting server! Why you ask?

Biggest reason is the WASTE of resources!

By preloading some site that has 5-10 unique visitors/day ( yes there are sites like that ), you are efectively wasting resources on the server. Imagine what would happen if you had around 500-600 applications on the server, all preloaded? How much RAM would the machine need to have to keep all those applications preloaded? I guess the answer is : A LOT!

So, conclusion, when to use IIS Application Warm-Up?

Use it on your own machine! When you have a dedicated machine just for 1-2 websites.  You, and your users will benefit from preloaded applications. Everything will open more faster!

If you have a shared hosting machine, MY reccomendation is to NOT use the IIS Application Warm-Up extension!

Well…in the very beginning, my answer is pretty much simple : yes AND no!  Why you ask?  Just keep reading…

Compression, regardless if its the compression of static or dynamic content is using CPU – that is a fact we can all agree upon.

Lets say compression is a tradeoff – CPU for bandwidth..use more cpu to “burn” less bandwidth.

The big question here is how to decide if you will benefit from turning on compression or not.

Most of us have “multi-purpose” servers which run various roles at the same time ( database server ( MYSQL , MSSQL, etc…) , mail server, web server, etc.. and all of those services are spending a certain amount of CPU time.

By turning on the compression, you will automatically “lose” some CPU time which could have been used for faster execution of some database queries i.e…

This is the key problem in our case. Is your machine powerfull enough to handle all the services it is already running + compression? Because in the end, you could end up with an even slower server!!!

If you have a small server ( 1 x quad core cpu, 4-8 gb ram ) which is under heavy or moderate load and you want your applications to be “jumpy” ( i.e : fast editing of some articles in your favourite CMS system ),  or you have lots of queries on your database service: DONT use compression, save your CPU time.  If you have some CPU time to spare,  turn on compression of static content which doesn’t change a lot or only needs to be downloaded every once in a while ( browser cache expires )

In case you have a big machine ( 2x quad core nehalems, 24 gb ram, 4x enterprise level hdd’s with hw raid controller, etc…)  and you have CPU time to spare, by all means, turn on the compression of both static and dynamic content. You will notice great improvements on your bandwidth costs

In my experience, the biggest bottleneck on 99% of the server is the disk system. Ofcourse, in my case we are talking about some small RAIDs with 4-8 disks.

Great example of that case are these two pictures:

CPU usage on a 2xQUAD core server with 16 gb of RAM.

As we can see, the server peaks out at 28% of cpu usage, which means it has lots of CPU time to spare for compression.

However, lets have a look at its disk system:

As we can see on the graph, average DISK FREE TIME is 64%, which is a bit low for my personal taste. I would be much more comfortable that this number is around 70 -75% .

This machine would definatelly benefit from turning on compression, since it has lots of CPU time to spare.

But I own a small server…is there any hope for me???

NEVER FEAR IIS 7 IS HERE!!!

One great feature in IIS 7 and IIS 7.5 is that it  offers CPU roll-off function.  When CPU usage goes  above certain level you set,  IIS server will stop compression and when usage drops below the level you have set up, it will start compressing content again.  This function is controlled by the staticCompressionEnableCpuUsage and dynamicCompressionDisableCpuUsage attributes.

For the owners of smaller machines, this function is just AWESOME! It will save you from unpleasant suprises like getting a call at 2-3 AM with a complaint that the SQL server is very sloooow.

In the end; you be the judge. Will you turn on compression on your server or not? Let me know…

Hello after a very long pause . Continuing the “Setuping a server for web hosting” series with the new smashing video of how to install PHP on your machine! AWESOME right?

In the very begining of the post, lets learn what PHP is exactly.

So, Wikipedia says ( and I add that I agree with her :

PHP: Hypertext Preprocessor is a widely used, general-purpose scripting language that was originally designed for web development to produce dynamic web pages. For this purpose, PHP code is embedded into the HTML source document and interpreted by a web server with a PHP processor module, which generates the web page document. As a general-purpose programming language, PHP code is processed by an interpreter application in command-line mode performing desired operating system operations and producing program output on its standard output channel. It may also function as a graphical application. PHP is available as a processor for most modern web servers and as standalone interpreter on most operating systems and computing platforms.

PHP was originally created by Rasmus Lerdorf in 1995 and has been in continuous development ever since. The main implementation of PHP is now produced by The PHP Group and serves as the de facto standard for PHP as there is no formal specification.PHP is free software released under the PHP License.

So, now when we know what PHP is, lets check the basic PHP syntax:

<?php
    echo "Hello World";
   /* echo("Hello World"); works as well, although echo isn't a
    function (it's a language construct). In some cases, such
    as when multiple parameters are passed to echo, parameters
    cannot be enclosed in parentheses */
?>

You can find out more about the PHP project on www.php.net.

In this case, I will use Windows Web Platform Installer to install PHP on my machine. By going on this link you can use the installer to install PHP on your machine (same link is also used in the video).

Check out the video ( dont forget to put it in full screen mode :

So, now when you’re all full of excitement and thinking “LOL, installing PHP is extra easy, everyone could do it” ….NAH!!! and the host says: You lost

First, lets check it the PHP is working at all on our server. Yes, we could use the above stated syntax, but that just isnt fun . To get some detailed info on our PHP lets use this command:

<?php phpinfo(); ?>

Lets navigate to c:\inetpub\wwwroot\ and create a file called “phptest.php” . We will do that by clicking with the right mouse click , New – Text Document.  We will call it phptest.php.

Inside the document we will paste the above piece of code, save it and hold your fingers .

Point your browser to your server address, and to the phptest.php file.

If everything went smoothly you should see a page like this:

If you see the page that means that the PHP on our server is working properly.

Now, lets have some fun with security issues in PHP.

First of all, we have to check the php.ini file which is located in C:\Program Files (x86)\PHP\php.ini  . Lets open it with Notepad.

Most of the option are explained in the file itself, so we will skip them. Focus will be only on the most important ones.

One of the interesting lines is this :

; – magic_quotes_gpc = Off         [Performance]

As we can see, that option is commented out and DISABLED, and its best to leave it like that since its been deprecated in PHP 5.3.X .

Second option is:

register_globals = Off -

LEAVE THAT TURNED OFF. If you have clients who have ancient applications, tell them to rewrite them .  More about the discussion on using register globals can be found on this link

Third one that should be disabled is :

allow_url_fopen = On

Standard setting in the instllation is “On”, which is bad and wrong for many reasons . This setting should be turned of because of cross site scripting and security holes in lots of CMS/Forum scripts. When fopen is turned on, a potential attacker has an oportunity to use some security hole in your application to include his own file and gain access over your entire account. I really think you dont want that

But, never fear , alternative is here .Tell your customers or yourself to use curl for fetching remote content.

Standard fopen function should be replaced with a curl function :

Fopen function

    <?
    $result = fopen($url,r);
    ?>

Curl function

<?
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, 0);
$result = curl_exec($ch);
curl_close($ch);
?>

Next very important setting is :

disable_functions =

By standard, its empty…

Here are some of the most common functions to be disabled : system, exec, shell_exec, passthru, set_time_limit, error_log, ini_alter, dl, pfsockopen, openlog, syslog, readlink, symlink, link, leak, fsockopen, popen, escapeshellcmd, apache_child_terminate apache_get_modules, apache_get_version, apache_getenv, apache_note,apache_setenv,virtual

Google a bit to find what these functions do, but for an example, I would disable the “exec” command ( among others) , but you find out why
( hint : net user /add useraccountname mypassword )

When you’re done setting the php.ini file, all you need to do is restart IIS server and thats it.

Next step is to install some PHP based application and admire your work.

Ofcourse, this is a very fast tutorial on installing PHP . Configuration of it is the trickiest part and you should use the official php site and google to help you with it!

Cheers, Alesandro

Found an awesome post on DotNetPanel blog so I had to repost it

One of the things you want to avoid when deploying an ASP.NET application into production is to accidentally (or deliberately) leave the <compilation debug=”true”/> switch on within the application’s web.config file.

Doing so causes a number of non-optimal things to happen including:

1) The compilation of ASP.NET pages takes longer (since some batch optimizations are disabled)

2) Code can execute slower (since some additional debug paths are enabled)

3) Much more memory is used within the application at runtime

4) Scripts and images downloaded from the WebResources.axd handler are not cached

This last point is particularly important, since it means that all client-javascript libraries and static images that are deployed via WebResources.axd will be continually downloaded by clients on each page view request and not cached locally within the browser.  This can slow down the user experience quite a bit for things like Atlas, controls like TreeView/Menu/Validators, and any other third-party control or custom code that deploys client resources.  Note that the reason why these resources are not cached when debug is set to true is so that developers don’t have to continually flush their browser cache and restart it every-time they make a change to a resource handler (our assumption is that when you have debug=true set you are in active development on your site).

When <compilation debug=”false”/> is set, the WebResource.axd handler will automatically set a long cache policy on resources retrieved via it – so that the resource is only downloaded once to the client and cached there forever (it will also be cached on any intermediate proxy servers).  If you have Atlas installed for your application, it will also automatically compress the content from the WebResources.axd handler for you when <compilation debug=”false”/> is set – reducing the size of any client-script javascript library or static resource for you (and not requiring you to write any custom code or configure anything within IIS to get it).

What about binaries compiled with debug symbols?

One scenario that several people find very useful is to compile/pre-compile an application or associated class libraries with debug symbols so that more detailed stack trace and line error messages can be retrieved from it when errors occur.

The good news is that you can do this without having the have the <compilation debug=”true”/> switch enabled in production.  Specifically, you can use either a web deployment project or a web application project to pre-compile the code for your site with debug symbols, and then change the <compilation debug=”true”/> switch to false right before you deploy the application on the server.

The debug symbols and metadata in the compiled assemblies will increase the memory footprint of the application, but this can sometimes be an ok trade-off for more detailed error messages.

The <deployment retail=”true”/> Switch in Maching.config

If you are a server administrator and want to ensure that no one accidentally deploys an ASP.NET application in production with the <compilation debug=”true”/> switch enabled within the application’s web.config file, one trick you can use with ASP.NET V2.0 is to take advantage of the <deployment> section within your machine.config file.

Specifically, by setting this within your machine.config file:

<configuration>

<system.web>

<deployment retail=”true”/>

</system.web>

</configuration>

You will disable the <compilation debug=”true”/> switch, disable the ability to output trace output in a page, and turn off the ability to show detailed error messages remotely.  Note that these last two items are security best practices you really want to follow (otherwise hackers can learn a lot more about the internals of your application than you should show them).

Setting this switch to true is probably a best practice that any company with formal production servers should follow to ensure that an application always runs with the best possible performance and no security information leakages.  There isn’t a ton of documentation on this switch – but you can learn a little more about it here.

So, dear developers, keep on eye on debug state…it will make you and the server happier

ALL CREDITS OF THIS TEXT GO TO DOTNETPANEL BLOG!

Source : DotNetPanel Blog

Microsoft SQL Server 2005/2008 Express is a free edition of SQL Server that is ideal for learning, developing and powering desktop, web and small server applications, and for redistribution by ISVs.

Top Features

First, we need to have a query that will backup all databases to one folder… Well, here it is


DECLARE @name VARCHAR(50) -- database name
DECLARE @path VARCHAR(256) -- path for backup files
DECLARE @fileName VARCHAR(256) -- filename for backup
DECLARE @fileDate VARCHAR(20) -- used for file name
SET @path = 'E:\sqlbackups\' --path to the backup folder
SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112)
DECLARE db_cursor CURSOR FOR
SELECT name
FROM master.dbo.sysdatabases
WHERE name NOT IN ('tempdb')
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO @name
WHILE @@FETCH_STATUS = 0
BEGIN
SET @fileName = @path + @name + '_' + @fileDate + '.BAK'
BACKUP DATABASE @name TO DISK = @fileName
FETCH NEXT FROM db_cursor INTO @name
END
CLOSE db_cursor
DEALLOCATE db_cursor

This will backup everything except the “Temdb” database. So, lets take a look at the current state of our backup folder.

As we can see, only the backups from 21.11.2009 exist. So, what happens when we run the query using SQL Management Studio.

Here’s the query pasted in “New query window”.

Sooo…if everything went smoothly, we should get a message similar to this one :

Awesome! Now, lets take a look at our backup folder, are there any changes? Sure hope there is, because this post will totally suck if the backup didn’t complete

As we can see, this post will not suck after all Notice the new files, created on the 22.11.2009 . IT WORKS! All we have to do now is to automate the backup, so we don’t have to run the query everyday by hand.

We will do that using Task Scheduler (or Scheduled tasks in Windows 2003). We have to add a task to execute everyday in 3 AM.

The command we have to run is:

“C:\Program Files (x86)\Microsoft SQL Server\90\Tools\Binn\SQLCMD.EXE” -i “c:\pathtosqlquery\sqlbackup.sql”

Where “sqlbackup.sql” is the name of the file you have pasted the code mentioned before.

Here are a couple of pictures so you can see how to do it:

Now, click on “Create task”. That will click will start the task wizard. Enter the name of the task ( i.e “SQL Backups”.

After entering the name and choosing the option “Run wheneter user is logged on or not”, click on “Triggers”.

Click on “New”. It will bring up a new window

Choose the “Daily” option, and set the time when you want the task to run. I have set it to 3 AM .

ONE IMPORTANT THING!!! Dont forget to check the “Enable” checkbox , otherwise, you will end up scratching your head and cursing Microsoft and uncle Bill hehe.

After that, click on “Actions”, where we will define the program that should be runned… Click on “New”…

Paste this into the “Program/Script” box.

“C:\Program Files (x86)\Microsoft SQL Server\90\Tools\Binn\SQLCMD.EXE”

NOTE! – path to the “SQLCMD.exe” may be different on your system!!!

All that is left to do is to “Add arguments”

-i “c:\pathtosqlqueryfile\sqlbackup.sql”

Click on “OK”, enter the system password and thats it. Enjoy your daily backup.

HINT!!!! If you’re lazy like I am, you can use a powershell script to delete backup files older than X days. I have also covered that with a blog post. More info about that on this link.

Bye bye!

Back by popular demand part 3 of the “setup my web server” tutorials

In this post we will cover the instalation of MySQL on a Windows 2008 R2 based server. The process is the same if you would have a Windows 2003 machine also.

At the beginning, lets see what MySQL is.

MySQL is a relational database management system (RDBMS)^. MySQL stands for “My Structured Query Language“. The program runs as a server providing multi-user access to a number of databases.

The project has made its source code available under the terms of the GNU General Public License, as well as under a variety of proprietary agreements. MySQL is owned and sponsored by a single for-profit firm, the Swedish company MySQL AB, now a subsidiary of Sun Microsystems. As of 2009^[update] Oracle Corporation began the process of acquiring Sun Microsystems; Oracle holds the copyright to most of the MySQL codebase.

MySQL is often used in free software projects which require a full-featured database management system, such as WordPress, phpBB and other software built on the LAMP software stack. It is also used in very high-scale World Wide Web products including Wikipedia, Google and Facebook^.

Many web applications use MySQL as the database component of a LAMP software stack. Its popularity for use with web applications is closely tied to the popularity of PHP, which is often combined with MySQL.

So, what is the process of installing MySQL on a Windows machine? Well, it’s pretty straight forward so there is no need to write step-by-step tuturial..look at the video rather than reading

Dont forget to choose HD and the “full screen” option.

Song in the video is from Thievery Corporation. Song name is : Lebanese Blonde