Showing posts with label System Center Orchestrator. Show all posts
Showing posts with label System Center Orchestrator. Show all posts

Wednesday, January 23, 2013

Orchestrator 2012 Web Service Request Issue

I've started running into an issue w/ external requests to start a runbook via the web service.  I'm interested to see if others are experiencing the same issue.

It seems sporadically that requests coming in (w/ or w/o parameters) will fail and the web service will return a 405 (Method Not Allowed) status code with the response.  Trying the same request again moments later would succeed.

Turning on Failed Request Tracing (FRT) in IIS for the "Microsoft System Center 2012 Orchestrator Web Service" will reveal more details w/ the error. 

From the site highlighted in IIS, you can enable FRT from the Actions pane.  After FRT is enabled, you can create the rule to capture specific status codes (400,404,405,500 in the example below).







After the issue occurs and the request fails w/ a code of 405, a log file will be generated in the following folder if you accepted the default path - C:\inetpub\logs\FailedReqLogFiles\W3SVC2.  Towards the bottom of the log file, you'll find this error information:

<Data Name="Buffer">&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; standalone=&quot;yes&quot;?&gt;
&lt;error xmlns=&quot;http://schemas.microsoft.com/ado/2007/08/dataservices/metadata&quot;&gt;
  &lt;code&gt;&lt;/code&gt;
  &lt;message xml:lang=&quot;en-US&quot;&gt;The requested operation requires Publish permissions on the Runbook&lt;/message&gt;
&lt;/error&gt;</Data>

This appears to be a bogus error since sending the same request again will succeed using the same credentials.

After a little more digging, the error occurring seems to correlate w/ the ClearAuthorizationCache maintenance task in the SQL database.  Since this task computes/populates the folders, runbooks, and permissions....it theoretically make sense that while that task is running, incoming requests would fail since it thinks the user does not have permissions to the runbook (when it does!).  This could technically also affect requests being sent through the Orchestration Console to stop/start jobs.  By default, this task runs every 600 seconds (10 minutes).  I'm not sure of the duration it takes for the ClearAuthorizationCache task to complete, but it would depend on how much data there is to process (# of runbooks, folders, etc. in the environment).


If you have seen or do experience similar issues, I'd appreciate if you left a comment.  I also suggest opening a case w/ Microsoft to determine if this is a general product issue.

Wednesday, November 21, 2012

Run .NET Script: Catching PowerShell Output Into the Current Session

Here is a handy tip for catching PowerShell output into a variable as Published Data.

There are often times you need to execute a cmd within PowerShell that creates output in the cmd's output, but not PowerShell.

I came across a similar issue on the technet forums.
http://social.technet.microsoft.com/Forums/en-US/scogeneral/thread/a9511617-c174-4a5b-a531-3110471c9222

For example....

Running w/ PowerShell in the Run .NET Script activity cannot catch the output of the winrs cmd.  Even w/ adding the $Output = ... and adding the Output variable to published data.

$Output = winrs -r:server_name -u:server_name\administrator -p password net localgroup Administrators Domain\User /ADD

This will result in the Output variable in the published data being empty.

The trick is simply adding "2>&1" (without quotes) at the end of the cmd.  So the complete command would look like this.

 $Output = winrs -r:server_name -u:server_name\administrator -p password net localgroup Administrators Domain\User /ADD 2>&1

This will result in the $Output variable catching the result from winrs into the published data.  Also note, you may have to Flatten the data since the result may end up in multiple lines.




Here is a link that explains different methods for catching output in PowerShell.
http://mctexpert.blogspot.com/2010/11/what-does-2-mean-in-powershell.html

Wednesday, August 29, 2012

Lookup Where Variables Are Used in Runbooks and Activities

********Updated 8/31
 Added code to exclude searching object/policy instance tables to reduce time the query runs.

 AND (@TableName != '[dbo].[OBJECTINSTANCEDATA]') AND (@TableName != '[dbo].[OBJECTINSTANCES]') AND (@TableName != '[dbo].[POLICYINSTANCES]')
********

I've seen many people (including myself) submit enhancement requests for the product to include some sort of functionality of variable "mapping" to runbooks and activities.

With over  2600 variables in my production database, it can be very time consuming to track down everywhere variables are used and to determine if they can be deleted if no longer used....

Well....after some time of beating my head against a wall w/ SQL, I've finally gotten a working query to find the RunbookPath, RunbookName, ActivityName, and Table.Column (ActivityType.Field) where the variable is used in an Orchestrator instance!!!!

For now, this will find where variables are used.  I'd like to update the query to also be able to find counter and schedule instances as well.  This should also be dynamic to find variables used in all Integration Packs (including OIT).

I found a query to search across all tables and columns for a specific string on this site that got me in the right direction.
http://vyaskn.tripod.com/search_all_columns_in_all_tables.htm

From there I edited it to also include the other tables/fields to make the data useful in regards to SCOrch.

You would need to edit the highlighted field below with the variable name you would like to search for.  The only caveat is there cannot be any duplicate variable names.  If there are duplicates, a few lines can be edited out.  Then you could set @VarID equal to the guid of the variable you're attempting to lookup.

You can use this query to lookup the guid  by variable name if there are duplicates:

Select VARIABLES.UniqueID
From VARIABLES
INNER JOIN OBJECTS ON OBJECTS.UniqueID = VARIABLES.UniqueID
Where OBJECTS.Name = 'MyVariableName' and OBJECTS.Deleted != 1


Please let me know if you find any bugs or run into any errors w/ the query.  Also, if anyone sees any room for improvement w/ the query (by no means am I an expert DBA :)).

Here is an example of the output from the query.



****Depending on the size of the database, this query may take several minutes to run.

Here is the query used to find variables throughout the database:

--Originally Written by: Narayana Vyas Kondreddi
--Modified By: Jon Mattivi
--Purpose: Search all tables and columns in the Orchestrator database to find variable instances

DECLARE @VarName nvarchar(100), @VarID nvarchar(100)
SET @VarName = 'MyVariableName'

SET @VarID = (Select VARIABLES.UniqueID
From VARIABLES
INNER JOIN OBJECTS ON OBJECTS.UniqueID = VARIABLES.UniqueID
Where OBJECTS.Name = @VarName and OBJECTS.Deleted != 1)

   
CREATE TABLE #Results (RunbookPath nvarchar(1000), RunbookName nvarchar(250), ActivityName nvarchar(250), [Table.Column] nvarchar(370))

SET NOCOUNT ON

DECLARE @TableName nvarchar(256), @ColumnName nvarchar(128), @SearchStr2 nvarchar(110)

SET  @TableName = ''
SET @SearchStr2 = QUOTENAME('%' + @VarID + '%','''')

WHILE @TableName IS NOT NULL
   
BEGIN
    SET @ColumnName = ''
    SET @TableName =
    (
        SELECT MIN(QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME))
        FROM     INFORMATION_SCHEMA.TABLES
        WHERE         TABLE_TYPE = 'BASE TABLE'
            AND    QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) > @TableName
            AND (TABLE_SCHEMA) = 'dbo'
            AND    OBJECTPROPERTY(
                    OBJECT_ID(
                        QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME)
                         ), 'IsMSShipped'
                           ) = 0
    )

    WHILE (@TableName IS NOT NULL) AND (@ColumnName IS NOT NULL) AND (@TableName != '[dbo].[OBJECTINSTANCEDATA]') AND (@TableName != '[dbo].[OBJECTINSTANCES]') AND (@TableName != '[dbo].[POLICYINSTANCES]') AND ((SELECT TOP 1 COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = PARSENAME(@TableName, 1) AND (COLUMN_NAME) = 'UniqueID' AND (DATA_TYPE) = 'uniqueidentifier' AND (TABLE_SCHEMA) = 'dbo') is not null)
           
    BEGIN
        SET @ColumnName =
        (
            SELECT MIN(QUOTENAME(COLUMN_NAME))
            FROM     INFORMATION_SCHEMA.COLUMNS
            WHERE         TABLE_SCHEMA    = PARSENAME(@TableName, 2)
                AND    TABLE_NAME    = PARSENAME(@TableName, 1)
                AND    DATA_TYPE IN ('char', 'datetime', 'decimal', 'int', 'money', 'ntext', 'nvarchar', 'varbinary', 'varchar')
                AND    QUOTENAME(COLUMN_NAME) > @ColumnName
        )
        IF @ColumnName IS NOT NULL
           
        BEGIN
            INSERT INTO #Results
            EXEC
            (
                'IF EXISTS (Select TOP 1 ' + @TableName + '.' + @ColumnName + 'From ' + @TableName + ' (NOLOCK) WHERE ' + @TableName + '.' + @ColumnName + ' LIKE ' + @SearchStr2 + ')' +
                'BEGIN ' +
                'SELECT Resources2.[Path], Policy.[Name], ActObj.[Name],''' + @TableName + '.' + @ColumnName + '''' +
                'FROM ' + @TableName + ' (NOLOCK) ' +
                'INNER JOIN [dbo].[OBJECTS] ActObj (NOLOCK) ON ' + @TableName + '.[UniqueID] = ActObj.[UniqueID]' +
                'INNER JOIN [dbo].[POLICIES] Policy (NOLOCK) ON ActObj.[ParentID] = Policy.[UniqueID]' +
                'INNER JOIN [Microsoft.SystemCenter.Orchestrator.Internal].[Resources] Resources2 (NOLOCK) ON Policy.[UniqueID] = Resources2.[UniqueId]' +
                'WHERE ' + @TableName + '.' + @ColumnName + ' LIKE ' + @SearchStr2 + 'and ActObj.[Deleted] != 1 ' +
                'END'
            )
        END
    END   
END

SELECT * FROM #Results
Order By RunbookPath
   
DROP TABLE #Results

Friday, August 24, 2012

Run Program Activity issue with Server 2008

****
Update 9/7/2012
The issue lies at the OS level w/ the UAC settings.  The changes reflect the differences from 2008 to 2008 R2....
i.e. 
Run all administrators in Admin Approval mode
Behavior of the elevation prompt for administrators in admin approval mode
****

****
Update 8/30/2012
This appears to only happen when using alternate credentials in the "Run As" field.  I've also repro'd this back in Opalis, so it's not something w/ the updated Run Program activity in SCOrch.  More troubleshooting to come....
****

Yesterday I came across an issue w/ the Run Program standard activity.  I have installed the RU2 hotfix release late in July to resolve the issue running the activity against Server 2003/2008/2008SP2.

The activity does run "successfully" in the console.  However, it does not seem to be running the specified command or program in an elevated UAC session.

Pure Output from running "ipconfig /flushdns" via Run Program on a 2008 server:

The requested operation requires elevation.


While expected behavior on a 2008 R2 server:
Windows IP Configuration

Successfully flushed the DNS Resolver Cache.



 I've also posted the question to the TechNet forums to see if this is a known issue since RU2.
http://social.technet.microsoft.com/Forums/en-US/scogeneral/thread/b03cb650-4abc-44a6-a097-9c1bf4d979a9


Saturday, July 14, 2012

Find Runbook/Parameter GUIDs for Web Service POST

I sometimes find it redundant looking up the runbook and parameter guids when creating a new powershell script to invoke a runbook from the web service.  Here is a SQL query to look up the runbook by name and every parameter that you would need to use the web service to start the runbook.

Select lower(POLICIES.UniqueID) as RunbookID, lower(CUSTOM_START_PARAMETERS.UniqueID) as ParameterID, CUSTOM_START_PARAMETERS.value
From POLICIES
INNER JOIN OBJECTS  on POLICIES.UniqueID = OBJECTS.ParentID
LEFT OUTER JOIN CUSTOM_START_PARAMETERS on OBJECTS.UniqueID = CUSTOM_START_PARAMETERS.ParentID
Where POLICIES.Name = 'My Runbook Name' and policies.deleted = 0


Note that the query uses the lower() function to force the guids to lowercase.  This is required for the parameters when invoking a runbook or you will receive a 500 Internal Server Error.  The runbook guid doesn't seem to matter if it's upper or lower case, but the parameters NEED to be in lowercase (bug??/feature??).

This will return the PolicyID (RunbookID), Parameter guid, and the name required to start the runbook from the web service.

Monday, July 2, 2012

Find Text Activity Alternative

I've encountered an issue a few times now where I need to use the Find Text activity to search a rather large (~20MB) txt or log file for a specific string.  The PolicyModule process would spike to around 25% proc utilization for every Runbook Instance running.  Set the job concurrency to four or more instances and you can do the math to see this becomes an issue :).  On top of that....the activity would take around 20 minutes to complete for each instance.

Once again PowerShell comes to the rescue.  You can easily port the activity over to the Run .Net Script activity and set the Published Data field for "result" using the below example.

$file = "C:\temp\myfiletosearch.txt"
$searchtext = "Specific string to find here"

$content = Get-Content -Path $file | Select | Where {($_ -like "*$searchtext*")}

If ($content -ne $null)
{
    $result = "Text Found"
}
Else
{
    $result = "Text NOT Found"
}

Tuesday, June 19, 2012

FTP Integration Pack Update v1.2

****Update 20120918****
I've just released v1.3 of the FTP/SFTP IP.
Post:
http://jmattivi.blogspot.com/2012/09/scorch-ftp-integration-pack-update-v13.html
CodePlex:
https://scorchestrator.codeplex.com/releases/view/94807


 ****Update****
I've updated the IP to v1.2 to fix an issue w/ the List Folder activity when using Secure = False.  Updated build has been posted to codeplex.
https://scorchestrator.codeplex.com/releases/view/89913


I've kicked the tires on the FTP Integration Pack a bit more and have updated the List Folder activity.  It will now only output the directory contents rather than displaying the entire verbose output (which is kinda redundant from the results published data).

I've also updated the download to Stable from Beta.  Please let me know if any bugs are encountered and/or if there are any feature requests.

Unfortunately, the FTPWebRequest class doesn't support wildcards for Secure = False (FTP). However, the Secure = True method (SFTP) in the IP does support using wildcards such as * or *.txt.

One way to get around the FTP method using wildcards would be to use the List Folder activity to find the files in the folder, parse them, and then pass each into the Download File activity.

For instance, use List Folder to get the files in the Output published data:
07-12-12 08:56AM 0 asdf.txt
07-12-12 08:56AM 0 blah.txt

Parse the Output and pass asdf.txt and blah.txt to the Download File activity.


The Integration Pack can be downloaded from this link below:
https://scorchestrator.codeplex.com/releases/view/89913



Tuesday, June 12, 2012

Integration Pack for FTP

****Update 20120918****
I've just released v1.3 of the FTP/SFTP IP.
Post:

http://jmattivi.blogspot.com/2012/09/scorch-ftp-integration-pack-update-v13.html
CodePlex:

https://scorchestrator.codeplex.com/releases/view/94807

********Please see this post for the updated v1.2
http://jmattivi.blogspot.com/2012/06/ftp-integration-pack-update-v11.html
********

I've just published an FTP/SFTP Integration Pack that can be downloaded from https://scorchestrator.codeplex.com/releases/view/89427.

Here are the current activities supported in the IP.  The SFTP portion is based off of Putty's psftp software (documentation can be referenced here).  The FTP functionality is based off of the System.Net.FtpWebRequest Class (documentation can be referenced here).  I was looking to provide roughly the same functionality the FTP Integration Pack for Opalis had.



All fields are required for each activity.   Secure and AutoAcceptKey are set to True by default

Secure = True
 --SFTP

 Secure = False
 --FTP

AutoAcceptKey only applies if Secure = True
AutoAcceptKey  = True
--Automatically accept the host key presented by the server
****This can be insecure****

AutoAcceptKey = False
--This will technically cause the application to prompt to accept the key, which in turn would cause the activity to fail.  However, you can proactively manually store the host key and leave this set to false if security requires it.


Unfortunately, the FTPWebRequest class doesn't support wildcards for Secure = False (FTP). However, the Secure = True method (SFTP) in the IP does support using wildcards such as * or *.txt.

One way to get around the FTP method using wildcards would be to use the List Folder activity to find the files in the folder, parse them, and then pass each into the Download File activity.

For instance, use List Folder to get the files in the Output published data:
07-12-12 08:56AM 0 asdf.txt
07-12-12 08:56AM 0 blah.txt

Parse the Output and pass asdf.txt and blah.txt to the Download File activity.
 

Upload File Example:




Download File Example:



Please kick the tires on this and let me know if you find any bugs or see the potential for functionality improvements.

This is the first time I've really poked around w/ the OIT command line activity wizard....now I have to teach myself C# here shortly to start building projects in Visual Studio w/ the SDK going forward. :)