This will probably be a two-part post.
I recently made some enhancements to one of my scripts to take flattened published data and format it so that it's readable (while still flattened).
A little background where this might apply....
If you have many data values that need to be emailed out, it's not likely that you want to send an email for each value. Ideally, all of the values (filenames, database fields, etc.) would be emailed in a readable format in one email.
I most often use this PowerShell script immediately after a Query Database activity that has published data flattened seperated by "__%__" (without quotes). The Query Database activity would read multiple rows from a database that meet specified criteria. This particular example uses the Standard Logging IP by Charles Joy.
The published data passed into the Run .Net Script is the "Full line as string with fields separated by ';'".
Ex. Table Query Results:
Ex. Full line as string.... flattened with __%__ Published Data (not very pretty!)
6/14/2013 3:46:02 PM;3 - Failure;Server4;ActivityName: Copy File From A to B;ErrorSummary: Access Denied__%__6/14/2013 3:44:25 PM;3 - Failure;Server3;ActivityName: Copy File From A to B;ErrorSummary: File Does Not Exist__%__6/14/2013 3:42:07 PM;4 - Completed;Server2;Copy Completed;__%__6/14/2013 3:41:05 PM;4 - Completed;Server1;Copy Completed;
Posh Script to get the published data:
The Run .Net Script activity also has the published data flattened, but with line breaks. The table variable would then be sent in the email.
Ex.
6/14/2013 3:46:02 PM
3 - Failure
Server4
ActivityName: Copy File From A to B
ErrorSummary: Access Denied
6/14/2013 3:44:25 PM
3 - Failure
Server3
ActivityName: Copy File From A to B
ErrorSummary: File Does Not Exist
6/14/2013 3:42:07 PM
4 - Completed
Server2
Copy Completed
6/14/2013 3:41:05 PM
4 - Completed
Server1
Copy Completed
#######################################################################
$table = @()
$RawFailures = @'
<Published Data seperated by __%__>
'@
$Regex = [regex] '/*__%__'
$Failures = $Regex.Split("$RawFailures")
ForEach($Data in $Failures)
{
If ($Data.Split(";")[0].Length -ne 0) {
$Field1 = $Data.Split(";")[0]}
Else {$Field1 = $null}
If ($Data.Split(";")[1].Length -ne 0) {
$Field2 = $Data.Split(";")[1]}
Else {$Field2 = $null}
If ($Data.Split(";")[2].Length -ne 0) {
$Field3= $Data.Split(";")[2]}
Else {$Field3 = $null}
If ($Data.Split(";")[3].Length -ne 0) {
$Field4= $Data.Split(";")[3]}
Else {$Field4 = $null}
If ($Data.Split(";")[4].Length -ne 0) {
$Field5= $Data.Split(";")[4]}
Else {$Field5 = $null}
IF ($Data.Length -ne 0) {
$table += $Field1, $Field2, $Field3, $Field4, $Field5, ""
}
}
$table = @($table | Where-Object {$_ -ne $null})
#######################################################################
Showing posts with label Integration Pack. Show all posts
Showing posts with label Integration Pack. Show all posts
Friday, June 14, 2013
Friday, April 26, 2013
Schedule Activity: First Business Day of the Month
Maybe this is more straightforward than I originally thought it was....sometimes I tend to over think and complicate things than they actually are. So just in case someone else finds this helpful.... :)
I recently got a request for a workflow to run on the 1st calendar day of the month or first business day if the 1st fell on a weekend. I first though that it wasn't possible to do with the native scheduling and PowerShell was needed. After I wrote up a Posh script to do the heavy lifting, it dawned on me how to use the native schedule activity. It's not as straight forward as using the script, but does work.
Orchestrator's built-in schedule activity supports two options:
1) Specifying the calendar day(s) of the month to run
2) Specifying the week day to run
Here is the "main" runbook for the schedule that will invoke the rest of the workflow. It requires four different schedule activities to accomplish the logic. Each of the following links (green) have the logic "confirms to schedule equal to true".
Here is the PowerShell script to accomplish the same schedule result. The "$Success" variable would be in the Published Data to use in the link logic to continue or not if Success equals true.
$DayOfWeek = [DateTime]::Now.DayOfWeek
$DayOfMonth = [DateTime]::Now.Day
If ( (($dayOfMonth -eq 1) -and ($dayOfWeek -ge [DayOfWeek]::Monday) -and ($dayOfWeek -le [DayOfWeek]::Friday)) -or ((($dayOfMonth -eq 2) -or ($dayOfMonth -eq 3)) -and ($dayOfWeek -eq [DayOfWeek]::Monday)) )
{
$Success = $true
}
Else {
$Success = $false
}
I recently got a request for a workflow to run on the 1st calendar day of the month or first business day if the 1st fell on a weekend. I first though that it wasn't possible to do with the native scheduling and PowerShell was needed. After I wrote up a Posh script to do the heavy lifting, it dawned on me how to use the native schedule activity. It's not as straight forward as using the script, but does work.
Orchestrator's built-in schedule activity supports two options:
1) Specifying the calendar day(s) of the month to run
2) Specifying the week day to run
Here is the "main" runbook for the schedule that will invoke the rest of the workflow. It requires four different schedule activities to accomplish the logic. Each of the following links (green) have the logic "confirms to schedule equal to true".
Here is the PowerShell script to accomplish the same schedule result. The "$Success" variable would be in the Published Data to use in the link logic to continue or not if Success equals true.
$DayOfWeek = [DateTime]::Now.DayOfWeek
$DayOfMonth = [DateTime]::Now.Day
If ( (($dayOfMonth -eq 1) -and ($dayOfWeek -ge [DayOfWeek]::Monday) -and ($dayOfWeek -le [DayOfWeek]::Friday)) -or ((($dayOfMonth -eq 2) -or ($dayOfMonth -eq 3)) -and ($dayOfWeek -eq [DayOfWeek]::Monday)) )
{
$Success = $true
}
Else {
$Success = $false
}
Thursday, February 14, 2013
Orchestrator and Excel Automation
I figured I'd throw this out here in case anyone is looking to automate Excel tasks through Orchestrator and PowerShell and running into issues. I was tasked with modifying an Excel file (removing worksheets and saving as a csv file) in SCOrch as part of a workflow prior to processing the file.
I used the PowerShell script below in the Run .Net Script activity.
Script
#Remove Worksheets
$objExcel = new-object -Com Excel.Application
$xlFixedFormat = [Microsoft.Office.Interop.Excel.XlFileFormat]::xlWorkbookDefault
$objExcel.Visible = $false
$objExcel.displayalerts = $false
$objWorkbook = $objExcel.Workbooks.Open("\\PathtoMyFile\MyFile.xlsx")
$objWorksheets = $objWorkbook.Worksheets | Where {($_.Name -ne "Data")}
If ($objWorksheets) {
ForEach ($objWorksheet in $objWorksheets) {
$objWorksheet.Delete()
}
}
$objWorkbook.Save()
$objExcel.Workbooks.Close()
$objExcel.quit()
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($objExcel)
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($objWorkbook)
Remove-Variable objExcel
Remove-Variable objWorkbook
Running this script manually on the runbook server worked fine and as expected. However, running the script in SCOrch throws the following error (or similar error with opening the file).
Error
Exception calling "Open" with "1" argument(s): "Microsoft Excel cannot access the file '\\PathtoMyFile\MyFile.xlsx'. There are several possible reasons:
• The file name or path does not exist.
• The file is being used by another program.
• The workbook you are trying to save has the same name as a currently open workbook."
To solve this issue, create an empty folder called "Desktop" under the systemprofile folder as shown below:
I used the PowerShell script below in the Run .Net Script activity.
Script
#Remove Worksheets
$objExcel = new-object -Com Excel.Application
$xlFixedFormat = [Microsoft.Office.Interop.Excel.XlFileFormat]::xlWorkbookDefault
$objExcel.Visible = $false
$objExcel.displayalerts = $false
$objWorkbook = $objExcel.Workbooks.Open("\\PathtoMyFile\MyFile.xlsx")
$objWorksheets = $objWorkbook.Worksheets | Where {($_.Name -ne "Data")}
If ($objWorksheets) {
ForEach ($objWorksheet in $objWorksheets) {
$objWorksheet.Delete()
}
}
$objWorkbook.Save()
$objExcel.Workbooks.Close()
$objExcel.quit()
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($objExcel)
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($objWorkbook)
Remove-Variable objExcel
Remove-Variable objWorkbook
Running this script manually on the runbook server worked fine and as expected. However, running the script in SCOrch throws the following error (or similar error with opening the file).
Error
Exception calling "Open" with "1" argument(s): "Microsoft Excel cannot access the file '\\PathtoMyFile\MyFile.xlsx'. There are several possible reasons:
• The file name or path does not exist.
• The file is being used by another program.
• The workbook you are trying to save has the same name as a currently open workbook."
To solve this issue, create an empty folder called "Desktop" under the systemprofile folder as shown below:
C:\Windows\SysWOW64\config\systemprofile\Desktop
C:\Windows\System32\config\systemprofile\Desktop
Wednesday, November 21, 2012
Run .NET Script: Catching PowerShell Output Into the Current Session
Here is a handy tip for catching PowerShell output into a variable as Published Data.
There are often times you need to execute a cmd within PowerShell that creates output in the cmd's output, but not PowerShell.
I came across a similar issue on the technet forums.
http://social.technet.microsoft.com/Forums/en-US/scogeneral/thread/a9511617-c174-4a5b-a531-3110471c9222
For example....
Running w/ PowerShell in the Run .NET Script activity cannot catch the output of the winrs cmd. Even w/ adding the $Output = ... and adding the Output variable to published data.
$Output = winrs -r:server_name -u:server_name\administrator -p password net localgroup Administrators Domain\User /ADD
This will result in the Output variable in the published data being empty.
The trick is simply adding "2>&1" (without quotes) at the end of the cmd. So the complete command would look like this.
$Output = winrs -r:server_name -u:server_name\administrator -p password net localgroup Administrators Domain\User /ADD 2>&1
This will result in the $Output variable catching the result from winrs into the published data. Also note, you may have to Flatten the data since the result may end up in multiple lines.
Here is a link that explains different methods for catching output in PowerShell.
http://mctexpert.blogspot.com/2010/11/what-does-2-mean-in-powershell.html
There are often times you need to execute a cmd within PowerShell that creates output in the cmd's output, but not PowerShell.
I came across a similar issue on the technet forums.
http://social.technet.microsoft.com/Forums/en-US/scogeneral/thread/a9511617-c174-4a5b-a531-3110471c9222
For example....
Running w/ PowerShell in the Run .NET Script activity cannot catch the output of the winrs cmd. Even w/ adding the $Output = ... and adding the Output variable to published data.
$Output = winrs -r:server_name -u:server_name\administrator -p password net localgroup Administrators Domain\User /ADD
This will result in the Output variable in the published data being empty.
The trick is simply adding "2>&1" (without quotes) at the end of the cmd. So the complete command would look like this.
$Output = winrs -r:server_name -u:server_name\administrator -p password net localgroup Administrators Domain\User /ADD 2>&1
This will result in the $Output variable catching the result from winrs into the published data. Also note, you may have to Flatten the data since the result may end up in multiple lines.
Here is a link that explains different methods for catching output in PowerShell.
http://mctexpert.blogspot.com/2010/11/what-does-2-mean-in-powershell.html
Tuesday, September 18, 2012
SCOrch FTP Integration Pack Update v1.3
I've just released v1.3 of the Orchestrator IP for FTP/SFTP. You can find it on codeplex at this link:
Please install this in a test environment before deploying to production! Also, please comment on any issues/bugs you may encounter.
********
Since adding the extra "UsePassive" and "UseBinary" input fields, please note the following steps required for existing installs upgraded to v1.3. These fields must be commited to the database in the existing activity before they will work.
1) Stop runbooks where the activities are located.
2) Check out the runbook
3) Open the activity and click "Finish" (no changes required)
4) Check in the runbook
(Thanks to Jeff for the insight for commiting the added fields to existing activities!)
********
********
Since adding the extra "UsePassive" and "UseBinary" input fields, please note the following steps required for existing installs upgraded to v1.3. These fields must be commited to the database in the existing activity before they will work.
1) Stop runbooks where the activities are located.
2) Check out the runbook
3) Open the activity and click "Finish" (no changes required)
4) Check in the runbook
(Thanks to Jeff for the insight for commiting the added fields to existing activities!)
********
Activity Change Log:
Create Folder
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Delete File
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Delete Folder
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Download File
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for deleting source
file(s)
Add option for bin/ascii transfer
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
List Folder
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Rename File
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Upload File
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for deleting source
file(s)
Add option for bin/ascii transfer
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Empty File/0 byte Error Handling
Labels:
ftp,
Integration Pack,
IP,
Orchestrator,
Orchestrator Integration Toolkit,
Parameter,
PowerShell,
psftp,
runbook,
scorch,
System Center Orchestrator 2012,
System.Net.FtpWebRequest,
Variables,
workflows
Friday, August 24, 2012
Run Program Activity issue with Server 2008
****
Update 9/7/2012
The issue lies at the OS level w/ the UAC settings. The changes reflect the differences from 2008 to 2008 R2....
i.e.
Run all administrators in Admin Approval mode
Behavior of the elevation prompt for administrators in admin approval mode
****
****
Update 8/30/2012
This appears to only happen when using alternate credentials in the "Run As" field. I've also repro'd this back in Opalis, so it's not something w/ the updated Run Program activity in SCOrch. More troubleshooting to come....
****
Yesterday I came across an issue w/ the Run Program standard activity. I have installed the RU2 hotfix release late in July to resolve the issue running the activity against Server 2003/2008/2008SP2.
The activity does run "successfully" in the console. However, it does not seem to be running the specified command or program in an elevated UAC session.
Pure Output from running "ipconfig /flushdns" via Run Program on a 2008 server:
The requested operation requires elevation.
While expected behavior on a 2008 R2 server:
Windows IP Configuration
Successfully flushed the DNS Resolver Cache.
I've also posted the question to the TechNet forums to see if this is a known issue since RU2.
http://social.technet.microsoft.com/Forums/en-US/scogeneral/thread/b03cb650-4abc-44a6-a097-9c1bf4d979a9
Update 9/7/2012
The issue lies at the OS level w/ the UAC settings. The changes reflect the differences from 2008 to 2008 R2....
i.e.
Run all administrators in Admin Approval mode
Behavior of the elevation prompt for administrators in admin approval mode
****
****
Update 8/30/2012
This appears to only happen when using alternate credentials in the "Run As" field. I've also repro'd this back in Opalis, so it's not something w/ the updated Run Program activity in SCOrch. More troubleshooting to come....
****
Yesterday I came across an issue w/ the Run Program standard activity. I have installed the RU2 hotfix release late in July to resolve the issue running the activity against Server 2003/2008/2008SP2.
The activity does run "successfully" in the console. However, it does not seem to be running the specified command or program in an elevated UAC session.
Pure Output from running "ipconfig /flushdns" via Run Program on a 2008 server:
While expected behavior on a 2008 R2 server:
Windows IP Configuration
Successfully flushed the DNS Resolver Cache.
I've also posted the question to the TechNet forums to see if this is a known issue since RU2.
http://social.technet.microsoft.com/Forums/en-US/scogeneral/thread/b03cb650-4abc-44a6-a097-9c1bf4d979a9
Tuesday, June 19, 2012
FTP Integration Pack Update v1.2
****Update 20120918****
I've just released v1.3 of the FTP/SFTP IP.
Post:
http://jmattivi.blogspot.com/2012/09/scorch-ftp-integration-pack-update-v13.html
CodePlex:
https://scorchestrator.codeplex.com/releases/view/94807
****Update****
I've updated the IP to v1.2 to fix an issue w/ the List Folder activity when using Secure = False. Updated build has been posted to codeplex.
https://scorchestrator.codeplex.com/releases/view/89913
I've kicked the tires on the FTP Integration Pack a bit more and have updated the List Folder activity. It will now only output the directory contents rather than displaying the entire verbose output (which is kinda redundant from the results published data).
I've also updated the download to Stable from Beta. Please let me know if any bugs are encountered and/or if there are any feature requests.
Unfortunately, the FTPWebRequest class doesn't support wildcards for Secure = False (FTP). However, the Secure = True method (SFTP) in the IP does support using wildcards such as * or *.txt.
One way to get around the FTP method using wildcards would be to use the List Folder activity to find the files in the folder, parse them, and then pass each into the Download File activity.
For instance, use List Folder to get the files in the Output published data:
07-12-12 08:56AM 0 asdf.txt
07-12-12 08:56AM 0 blah.txt
Parse the Output and pass asdf.txt and blah.txt to the Download File activity.
The Integration Pack can be downloaded from this link below:
https://scorchestrator.codeplex.com/releases/view/89913
I've just released v1.3 of the FTP/SFTP IP.
Post:
http://jmattivi.blogspot.com/2012/09/scorch-ftp-integration-pack-update-v13.html
CodePlex:
https://scorchestrator.codeplex.com/releases/view/94807
****Update****
I've updated the IP to v1.2 to fix an issue w/ the List Folder activity when using Secure = False. Updated build has been posted to codeplex.
https://scorchestrator.codeplex.com/releases/view/89913
I've kicked the tires on the FTP Integration Pack a bit more and have updated the List Folder activity. It will now only output the directory contents rather than displaying the entire verbose output (which is kinda redundant from the results published data).
I've also updated the download to Stable from Beta. Please let me know if any bugs are encountered and/or if there are any feature requests.
Unfortunately, the FTPWebRequest class doesn't support wildcards for Secure = False (FTP). However, the Secure = True method (SFTP) in the IP does support using wildcards such as * or *.txt.
One way to get around the FTP method using wildcards would be to use the List Folder activity to find the files in the folder, parse them, and then pass each into the Download File activity.
For instance, use List Folder to get the files in the Output published data:
07-12-12 08:56AM 0 asdf.txt
07-12-12 08:56AM 0 blah.txt
Parse the Output and pass asdf.txt and blah.txt to the Download File activity.
The Integration Pack can be downloaded from this link below:
https://scorchestrator.codeplex.com/releases/view/89913
Tuesday, June 12, 2012
Integration Pack for FTP
****Update 20120918****
I've just released v1.3 of the FTP/SFTP IP.
Post:
http://jmattivi.blogspot.com/2012/09/scorch-ftp-integration-pack-update-v13.html
CodePlex:
https://scorchestrator.codeplex.com/releases/view/94807
********Please see this post for the updated v1.2
http://jmattivi.blogspot.com/2012/06/ftp-integration-pack-update-v11.html
********
I've just published an FTP/SFTP Integration Pack that can be downloaded from https://scorchestrator.codeplex.com/releases/view/89427.
Here are the current activities supported in the IP. The SFTP portion is based off of Putty's psftp software (documentation can be referenced here). The FTP functionality is based off of the System.Net.FtpWebRequest Class (documentation can be referenced here). I was looking to provide roughly the same functionality the FTP Integration Pack for Opalis had.
All fields are required for each activity. Secure and AutoAcceptKey are set to True by default
Secure = True
--SFTP
Secure = False
--FTP
AutoAcceptKey only applies if Secure = True
AutoAcceptKey = True
--Automatically accept the host key presented by the server
AutoAcceptKey = False
--This will technically cause the application to prompt to accept the key, which in turn would cause the activity to fail. However, you can proactively manually store the host key and leave this set to false if security requires it.
Unfortunately, the FTPWebRequest class doesn't support wildcards for Secure = False (FTP). However, the Secure = True method (SFTP) in the IP does support using wildcards such as * or *.txt.
One way to get around the FTP method using wildcards would be to use the List Folder activity to find the files in the folder, parse them, and then pass each into the Download File activity.
For instance, use List Folder to get the files in the Output published data:
07-12-12 08:56AM 0 asdf.txt
07-12-12 08:56AM 0 blah.txt
Parse the Output and pass asdf.txt and blah.txt to the Download File activity.
Upload File Example:

Download File Example:
Please kick the tires on this and let me know if you find any bugs or see the potential for functionality improvements.
This is the first time I've really poked around w/ the OIT command line activity wizard....now I have to teach myself C# here shortly to start building projects in Visual Studio w/ the SDK going forward. :)
I've just released v1.3 of the FTP/SFTP IP.
Post:
http://jmattivi.blogspot.com/2012/09/scorch-ftp-integration-pack-update-v13.html
CodePlex:
https://scorchestrator.codeplex.com/releases/view/94807
********Please see this post for the updated v1.2
http://jmattivi.blogspot.com/2012/06/ftp-integration-pack-update-v11.html
********
I've just published an FTP/SFTP Integration Pack that can be downloaded from https://scorchestrator.codeplex.com/releases/view/89427.
Here are the current activities supported in the IP. The SFTP portion is based off of Putty's psftp software (documentation can be referenced here). The FTP functionality is based off of the System.Net.FtpWebRequest Class (documentation can be referenced here). I was looking to provide roughly the same functionality the FTP Integration Pack for Opalis had.
All fields are required for each activity. Secure and AutoAcceptKey are set to True by default
Secure = True
--SFTP
Secure = False
--FTP
AutoAcceptKey only applies if Secure = True
AutoAcceptKey = True
--Automatically accept the host key presented by the server
****This can be insecure****
AutoAcceptKey = False
--This will technically cause the application to prompt to accept the key, which in turn would cause the activity to fail. However, you can proactively manually store the host key and leave this set to false if security requires it.
Unfortunately, the FTPWebRequest class doesn't support wildcards for Secure = False (FTP). However, the Secure = True method (SFTP) in the IP does support using wildcards such as * or *.txt.
One way to get around the FTP method using wildcards would be to use the List Folder activity to find the files in the folder, parse them, and then pass each into the Download File activity.
For instance, use List Folder to get the files in the Output published data:
07-12-12 08:56AM 0 asdf.txt
07-12-12 08:56AM 0 blah.txt
Parse the Output and pass asdf.txt and blah.txt to the Download File activity.
Upload File Example:
Download File Example:
Please kick the tires on this and let me know if you find any bugs or see the potential for functionality improvements.
This is the first time I've really poked around w/ the OIT command line activity wizard....now I have to teach myself C# here shortly to start building projects in Visual Studio w/ the SDK going forward. :)
Subscribe to:
Posts (Atom)