Questions/General Comments/Requests

27 comments:

  1. Hi Jon,

    I wanted to shoot you a quick note because I'm hoping you can help me. I work for a company called Windward IT Solutions and we specialize in implementing SCORCH along with a few other workflow development tools. Right now we are in desperate need of another automation expert. I came across your blog and it evident that you have a lot of expertise with SCORCH and runbook automation.

    I was hoping you might be able to point me in the direction of a colleague that might be interested in the position and learning some of the other automation tools out there.

    I'd appreciate any help or suggestions you can provide as to how I can get in touch with some people that have experience with SCORCH. If this is something you're interested in as well I'd be happy to speak with you about the opportunity.

    Thanks,

    Chris Werner
    Technical Recruiter
    Windward IT Solutions
    703 812 0117

    ReplyDelete
    Replies
    1. Jon,

      Forgot to include my email address: cwerner@windwardits.com

      Delete
  2. Jon,

    Many thanks for the FTP Integration pack. Worked like a charm straight out of the box.
    Using it for pulling system logs over SSH.

    Thanks again
    Ben

    ReplyDelete
  3. We are using SCORCH 2012 R2. We still see some orphaned jobs on the Monitor runbooks.
    Is there any ways you know to fix it?
    We have already implemented the steps in this link
    http://www.concurrency.com/blog/how-to-clear-orphaned-orchestrator-runbook-instances/

    ReplyDelete
    Replies
    1. Hi,

      Can you confirm that you've actually stopped the running monitor runbooks with the orphaned jobs? The stored proc will only removed the orphaned instances if the runbook is NOT running. I don't think I've encountered a situation where the runbook has been stopped and the stored proc doesn't remove them.

      I've used these queries back in the Opalis days to clear up the orphaned instances (update statements being commented out).

      ****Find and remove orphaned policies****
      select *
      from dbo.policyinstances
      where timeended is null and status is null

      --update dbo.policyinstances
      --set [timeended] = getdate()
      --set [status] = 'warning'
      --where timeended is null and status is null


      ****Find and remove orphaned runbooks for specific runbook****
      select POLICYINSTANCES.UniqueID, POLICIES.Name
      from dbo.policyinstances, dbo.POLICIES
      where status is null and (POLICYINSTANCES.PolicyID = POLICIES.UniqueID) and POLICIES.Name = ''
      Order By PolicyID

      --update policyinstances
      --set policyinstances.[timeended] = getdate()
      -- ,policyinstances.[status] = 'warning'
      --From POLICYINSTANCES, POLICIES
      --where policyinstances.status is null and (POLICYINSTANCES.PolicyID = POLICIES.UniqueID) and POLICIES.Name = ''

      On a side note....You don't necessarily have to go into SQL to run the stored proc to clear orphaned instances. Right-clicking the management server in the Designer console and selecting "Log Purge" and Purge Now will also run that stored proc to remove the orphaned instances.

      Hope that helps....
      Jon

      Delete
    2. Forgot to mention....before running each of the queries above.

      1st Query - ALL runbooks need to be stopped prior to running.
      2nd Query - The particular runbook to clean up orphaned instances needs to be stopped.

      Delete
  4. Yes we had to stop runbooks and clear orphans.
    But is there a way to get permanent fix for this, does microsoft have a solution to avoid orphans?
    It is creasing our runbook servers sometimes, our runbooks stop working completely. Any ideas to fix it?

    ReplyDelete
    Replies
    1. Hi,

      Orphaned jobs are created (most of the time....) when runbook servers have connectivity issues with the database. The runbook server will go into a sleep mode for three minutes wherein all running runbooks will cease to run on that particular server and start running on another runbook server. After three minutes, the runbook server will continue to accept requests to run jobs.

      Orphaned instances don't really cause any performance issues (to my knowledge) other than being annoying while still visible.

      I don't foresee Microsoft changing any code to prevent the orphaned instances altogether....especially when the stored procedure is in the database to clear orphaned instances once the runbook is stopped.

      Jon

      Delete
  5. Hey Im back again. same person.. can you reply to the above comment if possible.
    http://jmattivi.blogspot.com/2012/06/high-availability-for-runbook-servers.html

    you have mentioned here the global variables have high availability.
    We have situation in SCORCH R2, where the jobs pile up.
    We have 8 runbooks servers. Each runbook server can take 100 jobs max load.
    When 3 runbook servers hit their max threshold, the performance of SCORCH becomes slow.
    hear teams mailing us alerts are not processed. can you help in this case?

    ReplyDelete
    Replies
    1. Hi,

      It's never a good thing when runbook servers are hitting their max limit.

      In this case, the solution would be to either

      Change the target runbook server (other than one of the three) for a bunch of the runbooks to spread the load out that they're not getting maxed out.
      OR
      Increase the max jobs for the runbook servers to something acceptable for your needs (above 100).

      Jon

      Delete
  6. Hi Jon,

    I was wondering if had any luck using the decrypt activity in orchestrator 2012? I configured it exactly how it is in opalis but it fails stating :

    Error Summary: public keyring not found

    Details:
    Secret keyring not found.

    regards,
    Jojo

    ReplyDelete
    Replies
    1. Hi Jojo,

      Does the passphrase contain any special characters? I ran into that issue a while ago that produced a similar error, I believe, was classified as a bug.

      Let me know,
      Jon

      Delete
    2. The passphrase does have a "*".

      Delete
    3. You could try creating a test set of keys to decrypt and create a passphrase with uppper and lower case characters (I think numbers were fine as well) to test. But I would guess that the "*" is creating the problem.

      Jon

      Delete
    4. I really would rather not pursue that since it would involve contacting each vendor we receive files from and the work to do that would be a major pain. Do you know of any other custom decrypt integration packs that I can try doing this with?

      Delete
    5. Technically, you would need to generate new keys and just provide the vendor the new public key to use going forward. The new passphrase would then be updated in the runbooks.

      Otherwise, I believe the only option would be to use the Run Program activity with gpg from the command line.

      Jon

      Delete
    6. I figured out how to get this to work. I isolated our secure and public ring files in one directory and pointed the decrypt activity keyring folder setting to that directory and that did the trick. Apparently opalis is smart enough to pair the appropriate files together to decrypt files but orchestrator isn't.

      Regards,
      Jojo

      Delete
    7. Awesome! Thanks for posting back! I can't imagine there were any changes to the PGP activities between Opalis and Orchestrator....but at least there's a workaround.

      Thanks again for posting back!!

      Jon

      Delete
  7. Hi Jon,

    I have 4 runbooks designed to handle quarterly audits, however I would like them to kick off automatically. I've tried using the monitor/check schedule, but to only start quarterly means I’m manually adding in exceptions (I’m trying to manual maintenance). One idea I had was to have my check schedule set on one day per month, then add an if statement in my first .Net script object (using Powershell). So if the current month variable doesn’t match the audit months, the whole runbook will stop. Hopefully this makes sense!

    Is there a way to stop a runbook from running if parameters aren’t meant?

    ReplyDelete
    Replies
    1. Hello,

      I wrote a post about scheduling a runbook to only run on the first business day of the month. You could use the same logic to specify (hard code) the months it's allowed to run using PowerShell. If it's not the correct month, the logic isn't met and the subsequent link would not let the runbook proceed.

      http://jmattivi.blogspot.com/2013/04/schedule-workaround-first-business-day.html

      Let me know if that helps. As far as stopping a runbook if parameters aren't met....do you literally mean if parameters in the initialize activity are not what is expected? For that you would also need PowerShell or the compare data activity to validate the parameters.

      Jon

      Delete
  8. Hi Jon,

    My company is tranfering many important files with Orchestrator 2012 R2 and we are using many different systems such as Windows, Linux, Solaris... For Linux/Solaris file transfers we are using the v1.3 FTP IP that you've made, but we have a problem. The List Folder is showing everything in a folder, (command ls -l) and we just need it to show only filenames, no dates or permissions, is there any workaround for this?

    Kind regards,
    Leon

    ReplyDelete
    Replies
    1. Hi Leon,

      The List Folder activity is pretty limited as it is just doing a "dir $Path". I just tested trying to include a switch prefixing the path in the activity, but it gets interpreted as part of the path. If you're strictly connecting to internal systems, you could alternatively use the Run SSH Command to replace the List Folder activity. In that you could specify to return only files.

      Jon

      Delete
  9. Thanks for your fast reply! I will try using the Run SSH Command.

    Leon

    ReplyDelete
  10. I have an update for your script to get jobs that are running. Would like to send it to you. How can I do that?

    ReplyDelete
    Replies
    1. Hi Mike,

      Do you have a github repo that you can put it in and share the repo link?

      I've also changed my method for querying running/pending jobs quite a bit since that post :)

      Thanks!
      Jon


      $url = "https://webservice.domain.net:8443/Orchestrator2012/Orchestrator.svc/Jobs()?`$filter=(Status eq 'Running' or Status eq 'Pending')&`$select=Id,RunbookId,LastModifiedTime,Status"

      $results = Invoke-RestMethod -Method Get -Uri $url -Credential $creds -ContentType "application/atom+xml" -UseBasicParsing

      $runbooks = @()

      foreach ($result in $results)
      {
      Clear-Variable -Name runbook -ea SilentlyContinue
      $runbook = New-Object psobject
      $runbook | Add-Member -MemberType NoteProperty -Name Id -Value $($result.content.properties.Id."#text")
      $runbook | Add-Member -MemberType NoteProperty -Name RunbookId -Value $($result.content.properties.RunbookId."#text")
      $runbook | Add-Member -MemberType NoteProperty -Name LastModifiedTime -Value $($result.content.properties.LastModifiedTime."#text")
      $runbook | Add-Member -MemberType NoteProperty -Name Status -Value $($result.content.properties.Status)

      $runbooks += $runbook
      }

      Delete
    2. Hi Jon, Funny I never saw your reply and Do not even know where I put this updated script. LOL But Hey I do have a question. Do you have an updated script for STOPPING a Runbook. OMG this one is killing me. I have been trying to use the code you have posted in the blog but I am getting 404 error from the server for some reason. I am able to get (using your other code) the JobID, RunbookID, Status, and LastModifiedTime for Job in question but when I pass that to your code to stop the running job I get:
      Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (400) Bad Request."

      Sure hope you have some other code laying on the floor... MS did not make this API friendly.

      I am using:
      $JobID = "f6e792a2-df66-4046-ba3d-aaf00b382411"
      $RunbookID = "0241f763-8220-4544-adc3-0c8edc1df81f"
      $LastModifiedTime = "2022-09-26T20:09:37.0533333"

      $url = "http://$($env:COMPUTERNAME):81/Orchestrator2012/Orchestrator.svc/Jobs(guid'$JobID')"
      $request = [System.Net.HttpWebRequest]::Create($url)
      $request.Credentials = $creds
      $request.Timeout = 60000
      $request.Accept = "application/atom+xml,application/xml"
      $request.Headers.Add("Accept-Charset", "UTF-8")
      $request.Headers.Add("X-HTTP-Method", "MERGE")
      $request.Headers.Add("If-Match", 'W/"datetime' + "'" + $LastModifiedTime + "'" + '"')
      $request.ContentType = "application/atom+xml"
      $request.Method = "POST"

      $requestBody = '' + "`n" +
      '' + "`n" +
      '' + "`n" +
      "" + "`n" +
      '' + $JobID + '' + "`n" +
      '' + $RunbookID + '' + "`n" +
      "Canceled" + "`n" +
      "" + "`n" +
      "" + "`n" +
      ""

      $requeststream=new-object System.IO.StreamWriter $request.GetRequestStream()
      $requeststream.Write($requestBody)
      $requeststream.Flush()
      $requeststream.Close()

      $response=$request.GetResponse()
      $requestStream=$response.GetResponseStream()
      $readStream=new-object System.IO.StreamReader $requestStream
      $Output=$readStream.ReadToEnd()

      start-sleep 5
      $readStream.Close()
      $response.Close()

      Thanks.. Mike

      Delete
    3. Hi Mike,

      Does the runbook actually stop? I thought this was a bug in the web service (which was later fixed in a rollup). The cancel/stop call did throw an error, but the runbook *does* actually stop.

      Jon

      Delete