I've just released v1.3 of the Orchestrator IP for FTP/SFTP. You can find it on codeplex at this link:
Please install this in a test environment before deploying to production! Also, please comment on any issues/bugs you may encounter.
********
Since adding the extra "UsePassive" and "UseBinary" input fields, please note the following steps required for existing installs upgraded to v1.3. These fields must be commited to the database in the existing activity before they will work.
1) Stop runbooks where the activities are located.
2) Check out the runbook
3) Open the activity and click "Finish" (no changes required)
4) Check in the runbook
(Thanks to Jeff for the insight for commiting the added fields to existing activities!)
********
********
Since adding the extra "UsePassive" and "UseBinary" input fields, please note the following steps required for existing installs upgraded to v1.3. These fields must be commited to the database in the existing activity before they will work.
1) Stop runbooks where the activities are located.
2) Check out the runbook
3) Open the activity and click "Finish" (no changes required)
4) Check in the runbook
(Thanks to Jeff for the insight for commiting the added fields to existing activities!)
********
Activity Change Log:
Create Folder
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Delete File
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Delete Folder
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Download File
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for deleting source
file(s)
Add option for bin/ascii transfer
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
List Folder
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Rename File
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Upload File
Resolve issue where host key is not
saved and cmd to change directory fails
Add option for deleting source
file(s)
Add option for bin/ascii transfer
Add option for passive/active
connection
Allow Paths/Filenames with spaces
Adjust FTP Timeout
Dir Change Error Handling
Empty File/0 byte Error Handling
Is there any way to use version 1.3 of this IP but to disable the ssh-2.00 requirement?
ReplyDeleteThere are still a lot of ssh-1.99 servers out there....
/Krister
Unfortunately no.
Delete"PSFTP uses the new SFTP protocol, which is a feature of SSH 2 only. (PSCP will also use this protocol if it can, but there is an SSH 1 equivalent it can fall back to if it cannot.)"
There must have been something else, because now when I try I can upload files to servers (answering with both 2.00 and 1.99)
DeleteAny chance you will add "change of filename" in upload/download file? so I can skip the rename activity in some of my runbooks.
//Krister
Great! Glad to hear it's working.
DeleteI know a few people have made requests for file renames within the download/upload activities....however, I most likely will not be updating the IP to include that feature. This would be a major rewrite to the existing activities when the functionality is present with the additional rename activity.
Since Microsoft has published their own FTP Integration Pack, I probably won't be doing any more feature additions to my IP and only bug fixes.
Have you tried the MS FTP IP? I tried it several times to upload on different ftp servers, and couldn't get the creation time correct. I always ended up with 1970-01-01... For "Create remote dir" I got the correct timestamp, but now for uploading files. Have you experienced that?
ReplyDeleteI decided to use your IP instead :)
//Krister
A little bit, enough to also find a potential bug in the List Folders/Files activity where it only works if the folder is empty and errors if there are one or more files present. That's using SFTP....haven't tried yet if the same things happens with a different protocol.
DeleteI'll poke around with the timestamps and post back the results. I didn't take notice when I was just testing functionality (basic uploading/downloading).
I was able to repro the time stamp for a downloaded file of being 1/1/1970 and 3/3/1970 for a file I uploaded.
DeleteHopefully MS will get the bugs cleaned up with this as more people begin to use the IP and report them. But I'm with you....for now I will be sticking with my IP for the time being. :)
When I get an error "Failed to Change Remote Directory!!!! " in the Upload File activity, I try to use the published variable Filename from Upload File to use in an error message in powershell, but it seems to be empty, is this something you can reproduce or is it my code?
ReplyDeleteMy upload is a specific file, not a wildcard
PowerShell code:
$subject="Error while uploading file {FileName from "Upload File"} to ftp server"
When I later use a SendEmail activity and use this published $subject the subject in the resulting email contains "Error while uploading file to ftp server"
//Krister
This is somewhat by design (similar to most if not all SCOrch, even standard, activities). When there is an error in the activity, the Error summary text contains the output from the activity and the regular published data is empty.
DeleteAs a workaround to supply the information in an error email, I've used the published data from a "Get File Status" activity upstream from the upload file activity or just subscribed to the variable that's used in the upload file activity (if it's always static).
Aah, ok nice to know. That's why the "Activity name" is empty if an activity fails sometimes. It could of course be set manually in the error message.
DeleteDo you know of any documentation that describes this?
//Krister
Well, the common logging published data should still be retrieved if an activity fails (including activity name). It's the activity specific data that is not published when an activity fails.
DeleteMaybe on those you didn't have common logging or subscribed to the "activity name" of the failed activity?
I'm not aware of any Microsoft documentation that points out when data is or is not published down the bus....
On a side note, in my comment above where I mentioned "regular" published data....I meant activity specific published data.
Jon,
DeleteI'm using your FTP pack because the MS FTP IP is failing to work with SFTP. However with yours, I'm repeatedly getting the "Failed to Change Remote Directory" error message, and I'm unable to figure out why. The account I am connecting to the remote FTP server with does have access and if I connect via Filezilla, I'm able to access all of the directories I need, however the Upload File activity fails every time with "Failed to Change Remote Directory". I'm also seeing several "unknown command "text" " lines in the error, with the text changing each time, sometimes saying "ally", others "y", others "This", and I am not sure of the reason for that, either.
Would you be able to shine some light on what I am experiencing? Thanks!
Hi gabriel,
DeleteHave you tried changing the remote directory or just using the default home dir for the user? Does it work on any other server that you are able to test?
I'm guessing it may somehow be a permissions issue. Can you try to download psftp.exe and test the same connection manually from the cmd line?
Please let me know the outcome.
The other unknown cmds are somewhat "by design" and won't have anything to do with the failure of the activity. The underlying code is setup to answer expected prompts from psftp. For instance, if you have auto accept key = true, it would anticipate the prompt and answer with "y". If you have already connected and accepted, the "y" response would throw the unknown command. The ensuing puts, gets, etc. would then run.
btw....just curious, what were you trying to do with Microsoft's FTP IP that was not working correctly? I was hoping to phase out my IP and use a MS supported IP....but I'm finding more reasons to make enhancements to mine and continue using it.
Hi Jon, thanks for the quick reply.
DeleteI'm currently converting a script I've been using for years which makes use of PSFTP to perform the uploads, so I know it works using PSFTP. However, traditionally I've made use of batch files instructing PSFTP to change directory (cd folder1, cd subfolder1, etc). I don't know if there is some format I need to use especially for your activity, so forgive me if I'm missing something. The target folder is nested two layers beneath Home (/home/folder1/folder2, I'm saving in folder2), so currently I have that whole path in the Remote Path value. Is there something else I need to do there? Unfortunately, saving in the Home directory is not an option.
I tried using the MS IP first to perform the same thing - connect to the remote server via SFTP (not FTPS) and upload a file to the specified directory. With your activity, it gives me error information, so I know I'm getting the remote directory change issue. With MS' activity, it simply failed and only listed the .Net exception information without giving any explanation for the failure. So I am not certain why it is failing, just that with the same credentials as I've always used with PSFTP and with your activity, it doesn't connect. I'm currently reaching out to MS and the Technet forums there to try and figure out what that issue is, but seeing as there are no other posts about SFTP on Technet, my hopes that someone else out there has gotten it to work and lived to tell the tale are not strong.
Correct, it should be changing the remote directory directly to the final path as you're doing. There really isn't anything special or tricky as far as specifying the path....Would you be able to share the full path of the remote directory you're putting in the DestPath field? Are there any special characters or anything? It will allow for spaces in the path....
DeleteI mean just for testing purposes - is there another folder on the server you're able to upload just to specify a different path?
It's connecting and then trying to change the remote directory, so the connection is being established.
Jon,
DeleteI determined the issue and the upload is now working successfully. The error was a foolish one - all of the documentation and the batch files used in the previous method listed the remote path as "\home\...", lowercase, while it is actually "\HOME\...", capitalized. It worked previously because the "cd" commands still identified the right folder to change the directory to, however I needed the remote path to have the correct case in order to get the activity to work in Orchestrator. Now I feel silly, but it's working, so there we go.
That's usually how it goes! :)
DeleteGlad you got it working!!
Definitely. Thank you very much for the help!
DeleteIn additional news, I ran a second test of the runbook using MS's Upload File activity to see if perhaps the same error prevented its usage - the answer is "no", as it still failed, citing "an error occurred during key exchange". Yours works, though, and that's all I need!
Hi Jon,
ReplyDeleteat the moment I'm onsite to customer.
He would like to try the FTP OIP ( FYI: The enviroment is SCORCH 2012 Sp1 )
My question is:
After registering the OIP v.1.3 we are unable to configure the module (In runbook designer, under Options, FTP configurations does not appear ).
No error on uploading the OIP and registering the module
Is it normal?
Davide
Hi Davide,
DeleteThat is correct. There is no configuration for the IP to setup in options. Each activity is configured individually.
Let me know if you have any other questions or concerns.
Jon
Hi Jon, thanks for Your reply.
DeleteThe problem is.. what about the configuration options for SFTP, where I can put the path for the certificate and the related password??
Are you trying to use FTPS - FTP over SSL (not SFTP - SSH FTP)?
DeleteUnfortunately, the IP only supports FTP and SFTP. Not FTPS.
Thanks Jon but I cannot ask the customer to change this way of life.
DeleteWe are tryin to implement SCORCH to use only one tool ( actually customer use task scheduler, winscp and so on ) ... Using the last version of deplyable OIP on Microsoft site we are able to configure all the Parameters I've indicated on my previous mail...
No problem, glad you got it working with the Microsoft IP. I was just pointing out that my IP only supports those two protocols (not to change the process to use one of them).
DeleteJust an fyi with the MS FTP IP....I know using the SFTP protocol, the IP had issues with downloading/uploading files and setting the created date to 1/1/1900 (or something like that). Also, the list folders/files activity did not work when the directory was not empty. This may not apply to other protocols in the IP, but just wanted to give you a heads up in case.
Hi Jon, I am using your IP and have a question is there anyway to change the port number such as 2222 or else during FTP transmission? My client has provided me their FTP server name, credential and restricted me to use port # 2222 in order to fetch the files successfully. Do you have any solution for this?
ReplyDeleteThank you,Hashmi
Hi Hashmi,
DeleteCan you try setting the server parameter to this below? You're basically adding " -P 822" immediately after the server name. Please let me know if this workaround is successful. Thanks!
servername.domain.com" -P 822"
Hello. I have intermitend error and the error is unable open file. I see too error log when show "this" command error.
ReplyDeleteHello,
DeleteThe unknown cmds are somewhat "by design" and won't have anything to do with the failure of the activity. The underlying code is setup to answer expected prompts from psftp. For instance, if you have auto accept key = true, it would anticipate the prompt and answer with "y". If you have already connected and accepted, the "y" response would throw the unknown command. The ensuing puts, gets, etc. would then run.
Which activity are you seeing the error with? Is the specific error returned stating "unable to open file"? Can you confirm the file exists and the account you're connecting with has appropriate permissions to the file?
Please let me know.
Hello,
ReplyDeleteA BIG thanks you for your IP.
It helped us on a SFTP upload because the IP from MS doest not support ssh-rsa and only support ssh-dss :( but yours, as it use psftp works with everythings :) .
Thanks again
Regards,
Neodolphin
Hi Neodolphin,
DeleteThanks for the comment! It's always reassuring the time and effort into the IP is helping others. I'm glad it's working for your requirements!
Jon
Hi Jon, here is Davide Again :)
ReplyDeletelast week I stayed to the customer where I had implemented Orchestrator. Again we decide to use Your FTP OIP implementation module. The FTP server is using SFTP protocol (FTP over SHH) and the module work as espected but... ( always there's a "but" ) we have noticed an unexpected behaviour.
We are going to upload 32 file and server and... on the log activity of the runbook we notice that the "upload file" is executed 32 times... What do You think about? Is this a bug or is this an expected behaviour? I've a repro on my local lab if U need..
Thanks
Davide
Hi Davide,
DeleteWhat is the activity preceding the upload file activity? If it is a "Get File Status" activity that finds 32 files, all of the following activities would be executed for each file in the Get File Status published data. This would be the expected behavior of Orchestrator. Are you passing in individual file names to the Upload File activity or are they wild carded in the file name parameter?
Please let me know the details of the runbook and if the above is accurate.
Thanks!
Jon
Hi Jon,thanks for your answer.
DeleteThis is the runbook activities detail:
- I'm using the data manipulation module to create a custom value for the data format (yyyymmdd)
- I'm using the published data from the data module to create a folder on a local server
- I use again the published data from the data module to create a folder to the ftp server (ftp over ssh) using "create folder" from your module
- I use copy file to copy 32 file ( 31 pdf and 1 txt ) froma remote server to the folder in local server
- I use upload file fo upload file from the local server to the ftp server using your module
As I told You everything works but on the runbook tester I'n noticed the strange behaviour to repeat upload file for 32 times
Thanks
Davide
Hi Jon, here is Davide again...
ReplyDeleteIn my last post i've forget to specify that I use wildcard to specify what file I would like to upload to the server.
To complete the test I've compiled the same runbook using the Microsoft module and a standard FTP server (no secure)
Thanks
Davide
Is the "Copy File" activity configured to have the published data flattened? If it is not flattened, the copy file is the activity that would cause the Upload File activity to run for each individual file that it copied. If both Copy and Upload are using wildcards, you should be able to flatten the Copy File activity as long as you're not subscribing to any published data from it.
DeleteJon
I think, we can enter masked file names like *.log on download file object.
ReplyDeletethanks
Hi Talha,
DeleteThat's correct (kind of). The SFTP (Secure = True) protocol supports wildcarded values in all activities. However, the FTP (Secure = False) does not support wild cards.
Jon
Hello I have this error and my passowrd is correct
ReplyDeleteFailed to Transfer!!!!
Looking up host "53.31.101.22" Connecting to 53.31.101.22 port 22
Server version: SSH-2.0-1.82_sshlib GlobalSCAPE
Using SSH protocol version 2
We claim version: SSH-2.0-PuTTY_Release_0.62
Doing Diffie-Hellman group exchange
Doing Diffie-Hellman key exchange with hash SHA-1
Host key fingerprint is:
ssh-rsa 2048 51:63:82:06:be:0d:49:74:69:19:90:3f:69:fe:78:1f
Initialised AES-256 CBC client->server encryption
Initialised HMAC-SHA1 client->server MAC algorithm
Initialised AES-256 CBC server->client encryption
Initialised HMAC-SHA1 server->client MAC algorithm
Using username "mpc_es_prod
".
Attempting keyboard-interactive authentication
Keyboard-interactive authentication failed
Access denied
Disconnected: No supported authentication methods available (server sent: )
Fatal: Disconnected: No supported authentication methods available (server sent: )
Exception: RuntimeException
Target site: PipelineBase.Invoke
Stack trace:
at System.Management.Automation.FlowControlNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
at System.Management.Automation.ParseTreeNode.Execute(Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)
at System.Management.Automation.StatementListNode.ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)
Hi,
DeleteGoing off of the "Access denied"....can you try connecting to the sftp server manually outside of Orchestrator (Filezilla)? This may be an issue with credentials, port, and server side authentication settings. Is the connection being made over port 22?
If you can successfully connect with the same username and password to the server over port 22 with Filezilla or some other app, you should be able to use the same with the integration pack.
Please let me know if you're successful or repro the same error outside of Orchestrator.
Jon
Hello. I connect with filezilla and psftp.
DeleteHmmmm....can you share the switches and values you're using in the psftp cmd (without the actual username and password)?
DeleteJust to confirm....you're using straight password authentication (not key auth) and port 22, correct?
Hello
ReplyDeleteI only open psftp connect to server with my user and password and accept key auth. I use por 22.
Do you have the following parameters set in the IP activity?
DeleteSecure = True
AutoAcceptKey = True
So when you run it manually....you're connection cmd looks exactly like this? If not, can you please share it?
psftp.exe -v -pw [Password] [Username]@[DestServer]
Then when prompted, you accept the server's key with "y"?
Are all of the activities giving the same result in the connection error?
Hello. Now seems this work. I have used variables for user and pasword. I think that have problems with copy paste and spaces. Thanks.
DeletePlease its possible use other port that 22. I have server with sftp in port 2222?
Great! Glad it's working!!
DeleteFor an alternate port, can you try this workaround?
Can you try setting the server parameter to this below? You're basically adding " -P 822" immediately after the server name. Please let me know if this workaround is successful. Thanks!
servername.domain.com" -P 822"
Hello.
ReplyDeleteWhem donwload activiy not found files to download this failed. is normal? How can avoid this result?
Hello,
DeleteCorrect....when the download file activity runs and it does not find any files matching to download, the activity will fail. This is the expected result.
You should build the logic into the link following the activity if you do not want that to be considered as a "failure". You can key off of the text in the Error Summary Text.
Please let me know if that makes sense.
Jon
Hello. But that string use for this? the error is the same for file not exist or for folder not exists. I think that if a file not exit not is motive for failed is better for example exist code 1 for example or other error control.
DeleteHello,
DeleteThe error messages will be different for a folder that doesn't exist compared to no files found.
If the folder doesn't exist, you should see a failed to change local or remote directory error.
If files aren't found, you should see a string that says "nothing matched".
Jon
Hello
ReplyDeleteThe problema is that the activiy always return failed.
Hi,
DeleteCorrect. For instance with the Download File activity....if there are no files found to download, the expected status result of the activity would be failed. If files are found and downloaded it's status will be success.
Depending on your requirements, if it is not considered an error when files are not found....you should update the link logic to key off of the string returned in the Error Summary Text.
Let me know if that makes sense or if you send an example error message I can give you a hand in updating the link logic.
Jon
Hello
Deletehow can i to do for that activity not report failed? it's posible? I control the link and create event with information with no files but the activity repot failed. I have reports for list activity with failed status and this show in the report.
Thanks in advanced
Hi,
DeleteUnfortunately, if there are no files available to upload/download, there is no way to prevent the activity from returning failed. The links can be configured to logically consider this as a success for the flow of the Runbook.
This is a general "issue"/design with Orchestrator and the way the status reporting is setup. I personally do not use or necessarily like the idea of using the activity status to report on successful or failed runbooks and/or workflows due to the fact a failed activity could be by design and mean the logic was successful.
Consider the Get File Status standard activity. If that does not find a file, it also fails the activity. Depending on the use, it may be considered a success not to find files.
Jon
Hello.
ReplyDeleteThanks for you help. Now i have other question. how can used this command put spidis.txt 'JI30A.AFD.J.DISPNAC(+1)' with the your integration pack? I need to do put and append file to mainframe host.
This work with ftp command but i don't now how to do with activity upload.
Hi,
DeleteAre you trying to append to the contents of the file or append something to the file name after it is uploaded?
Jon
I have both cases. I a simple put and i too need append
ReplyDeleteIt would most likely be easiest to append to the contents of the file prior to uploading it to the sftp server. You can rename a file after uploading it with the Rename File activity in the FTP IP. On the other hand, you could also rename the file as well before uploading it....whichever is easiest based on the workflow requirements.
DeleteHI Jon, here is Davide again
ReplyDeleteMay I ask U for something ( about SCORCH ) out of The SFTP module?
Hi Davide,
ReplyDeleteAbsolutely....what's up?
Jon
Are U ready? ;)
ReplyDeleteOk, look up to my previus request... Setting up the "Flatten" to the copy mail action resolve the problem for duplicate file action ( repeated 32 times ) but...after that action I need to delete the file into the origin folder ( and, If I use "Flatten" seems it does not work ).
Second step I need, after delete file, to send a mail with a list of the file from the origin folder. If copy file is not flatten
... the system sends 32 mail ( one for every row into the file)[I'm using the "standard" send mail action.
Have U same Idea?
NB
I'm installing an Exchange 2010 server to test the dedicate oip...
Does your workflow look like this?
DeleteCopy File > Upload File > Delete Source File > Send Email(with file list)
You would have to flatten the published data on each of the activities (copy, upload, and delete) and use the path and wildcard in each (not published data). If you choose to flatten the data using line breaks, you could subscribe to the file name from the Copy file to publish in the Send Email activity.
Does that make sense?
Jon
Hi Joh,
Delete..Yes , the workflow is more ore less like that but the problem is another one..
Customer use copy file with "*" to copy all the file into a remote folder then would like to read from this folder and obtain a list of the copied file and would like to delede exactly the list of the copied file. If I use the Flatten option I can obtain a list of file separated from a "break" or other symbol and the "delete" put the workflow in error state.
If I put the workflow in "flatten" mode I can resolve the duplicated activities of copy and upload and i can send only one mail but, because copy file is flatten, the delete file does not work (it seem it is not able to read or manage the text into the file )
Did U have some Idea?
PS.:
Using the dedicate Exchange IP does not resolve this "issue"
Please do not hesitate to contact me ( if u have skype my contact is davide.nando.brambilla ) for any question
Thanks
Davide
Hi Jon
DeleteI think i have solved my problem :)
I've implemented the Line Break run behavior and it works!
Thanks for all Your support!
Davide
Awesome! Glad you got it working!! :)
DeleteJon
Hi
ReplyDeleteI recently made a runbook that has to list a folder over sftp, grab the files if there is any, and then delete them.
But if the remote folder is empty i get this error:
Exception calling "Join" with "2" argument(s): "Value cannot be null.
Parameter name: value"
Is it something known, or am i doing it wrong?
Hi,
DeleteThat is the expected result of the List Folder activity when there are no files. The activity will result in a failed status if there are no files found.
The subsequent link following the List Folder would be used to control the logic on what action to take if there are or are no files found that match the filename mask.
I've personally only really used the list folder prior to an upload file activity to find any existing files and delete them (if they exist) prior to uploading new files. For downloading files, I've just used the download file activity w/o the list folder. If there are no files found to download, you can use the Output published data in a link to key off of the string "filename: nothing matched" (or something like that).
Let me know if that makes sense or helps.
Jon
The thing is, if the folder is empty, the runbook returns failed and stops right there.
DeleteHi,
DeleteIt fails on the list folder before the download, correct? In that case, the link after the List Folder activity would need to be updated to take the next step or task you would like to perform if there are no files.
Jon
I'm using v1.3 and is having an issue getting the download a new file to a directory that the file does not exist. The error is that it is trying to open for read and it does not exist.
ReplyDeleteHi,
DeleteCan you confirm that the account being used in the download file activity has permissions to download the file and that the SCOrch service account has modify permissions on the destination directory?
You could even test the transfer and account permissions using filezilla or psftp outside of Orchestrator.
Let me know.
Jon
Yes, I was able to transfer the file that does not exist in that directory using filezilla with the user account.
DeleteWere you using your account to run filezilla? That confirms the SFTP username has permisisons to the source file. I'm guessing at this point the Orchestrator service account does not have permissions to write the file to the destination directory.
ReplyDeleteIs the default of the download file activity to skip copying of the file or overwrite it if it exist?
ReplyDeleteBy default, it should overwrite the files.
DeleteI'm using v1.3 and are getting the following error using Download File activity:
ReplyDeleteFailed to Change Local Directory!!!!
Connected to Remote working directory is /prdctrl psftp> psftp: unknown command "System" psftp> Remote directory is now /edi/editst/job_scripts psftp> remote:/edi/editst/job_scripts/runbook_test.txt => local:runbook_test.txt Sent EOF message
Server sent command exit status 0
Disconnected: All channels closed
psftp>
Hi,
DeleteCan you confirm that the destination folder parameter is not empty and that the SCOrch service account has permissions to the directory?
Jon
Hi Jon I need to disturbe U once again for a problem with Orchestrator, it's something related to High availability configuration.. I really can't understand why if i implement multiple runbook server and they work at the same time. the runbook stop work... how many are u experienced on High Availability of this product??
ReplyDeleteThanks
Davide
Hi Davide,
DeleteCan you be more specific with your question? I'm not exactly sure what functionality or configuration you're asking in regards to HA.
Jon
Hi Jon, sorry for Delay
ReplyDeleteThe question is too easy but, i think, answer will be not (..). Customer has got big problem with High Availability on Orchestrator.
They have introduced a second runnbook server into the infrastructure but they are not enable to manage it when they try to switch runbook from one server to other. Unfortunately I have not found resources as setp-by-step how to implement and manage runbook in HA .
Davide
So they have added a second runbook server but runbooks are failing to run on the server? Does a runbook instance get created? What are the error messages seen?
DeleteHi, I am having minor issues connecting to a SFTP server running FileZilla
ReplyDeleteThe connection is always refused.
I am able to connect using WinSCP.
Is there a way that I can accept and save the SSL certificate?
Error Details:
File Failed to Transfer!!!!
Looking up host "myftpserver.com" Connecting to 10.0.0.254 port 22
Failed to connect to 10.0.0.254 : Network error: Connection refused
Network error: Connection refused
Fatal: Network error: Connection refused
Exception: RuntimeException
Target site: PipelineBase.Invoke
Stack trace:
at System.Management.Automation.FlowControlNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
at System.Management.Automation.ParseTreeNode.Execute(Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)
at System.Management.Automation.StatementListNode.ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)
Sean
Hi Sean,
DeleteAre you trying to use SFTP (ftp over ssh) or FTPS (ftp over ssl, requiring ssl certs) for the file transfer? My integration pack is only coded to use SFTP transfers. If that's the case, I believe Microsoft's FTP IP can perform the FTPS transfer or you could use WinSCP via the Run Program activity to do the transfer.
If you are trying to do an SFTP transfer, is it setup to use port 22?
Let me know,
Jon
Hello. Is posible use proxy with this integration pack? how or where set proxy?
ReplyDeleteHello,
DeleteIt would be great to get the answer, I'm looking for the same thing :)
Regards
Hi,
DeleteI don't think psftp supports using a proxy. I'll have to research and see if the FTP (unsecure) module can support using a proxy to code it in.
So, to answer your question....the IP does not currently support using a proxy.
Jon
This comment has been removed by the author.
ReplyDeleteHello,
ReplyDeleteI was directed to try you IP by microsoft support, butt it isn't working when I attempt to upload, I received the below "error summary text". I tried this with 2 different vendor sites. Any help you can provide is much appreciated.
regards,
Jojo
File Failed to Transfer!!!!
Looking up host "ftp.waproviderone.org
"
ssh_init: Host does not exist
Exception: RuntimeException
Target site: PipelineBase.Invoke
Stack trace:
at System.Management.Automation.FlowControlNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
at System.Management.Automation.ParseTreeNode.Execute(Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)
at System.Management.Automation.StatementListNode.ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context
Hi Jojo,
DeleteCan you confirm that there is not a space or return at the end in the "DestServer" parameter? I only ask since the end quote above that you posted is on a new line.
Does the server limit connections by IP address? If so and they haven't explicitly added your connecting IP, it would be blocked from connecting to the destination server.
Let me know,
Jon
Hello Jon,
DeleteThanks for this IP...i'm trying to use it to get a list of files from a particular folder and then subsequently delete files based on date. The Linux command equivalent would be something like this ; find /var/log/test/backup* -mtime +3 How could I accomplish the same using your IP? I don't see a way to filter based off date?
Hi,
DeleteSorry for the delayed response. Currently, there is no date filter for the list files activity. For that requirement, you could use the Run SSH Command activity with that line you have above, then use another activity to parse the results, and finally pipe those results into the FTP IP Delete Files activity (or another run ssh command activity to perform the deletes).
Jon
Hi Jon,
ReplyDeleteYour advise to use higher port worked fine. I used as you described " -P 2222" right after the server name in your FTP IP. Thank you Jon, and sorry to response you a bit late...Hashmi
Hi, I don't get that working. I am always getting this error message:
DeleteFailed to Change Remote Directory!!!!
psftp: no hostname specified; use "open host.name" to connect psftp> psftp: unknown command "y" psftp> psftp: unknown command "This" psftp> New local directory is psftp> psftp: not connected to a host; use "open host.name" psftp> psftp: not connected to a host; use "open host.name" psftp>
Exception: RuntimeException
Target site: PipelineBase.Invoke
Stack trace:
at System.Management.Automation.FlowControlNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
at System.Management.Automation.ParseTreeNode.Execute(Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)
at System.Management.Automation.StatementListNode.ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)
Any help please?
Hello,
DeleteCan you confirm you're including the quotes after the server name? From the error, it doesn't look like the hostname is being specified correctly.
Jon
First, thank you for the great IP! We have been using it for dozens of Orchestrator Runbooks now.
ReplyDeleteI am trying to download a mainframe (MVS) file. In MVS, the file name includes ' marks, for example 'FILE.123' and it causes problems, the error message is
At line:10 char:15
+ $FileName = ''FILE.123''
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
Unexpected token 'FILE.123''' in expression or statement.
downloading with Windows command line ftp works fine.
Could you please help?
Hello,
DeleteSince the underlying script is using single quotes for the filename, that's what is throwing it off. If you use additional single quotes around the filename (example below), it'll work. I just tested successfully.
''FILE.123''
This basically translates to '''FILE.123''' in the script preserving the single quotes in the variable. In hindsight, it should have used here strings for most if not all of the variables....
Let me know if that helps.
Jon
JES! It worked! There are no paths or folders while downloading files from MVS / mainframe, but it needs a dot to the path anyway:
DeleteSourcePath = .
FileName = ''FILE.123'' (not "FILE123")
Thanks Jon!
Best Regards,
Pekka
Hi,
ReplyDeleteis there a possibility to use public key authentication? I saw in the comments that they're injecting another port with using " -P 2222" (which is not working for me :( ) - but is there any chance to get public key up and running with -i ?
Or is there a easier way to get this?
Thanks
Currently, public key auth is not supported in the IP. I have posted the IP and source code to github where it could be modified pretty easily to run the code directly in the Run .NET Script activity.
Deletehttps://github.com/jmattivi/Orchestrator/tree/master/Integration%20Packs/FTP-SFTP
Jon
Hi Jon, Building a new Orchestrator server (2016) and wanted to download your IP for FTP as 2 of my runbooks use it on the old system. When I go to the https://archive.codeplex.com/?p=scorchestrator link and download the archive. It is a bunch of weird files. It doesn't appear to be a msi for install or an oip for import to deployment mgr. So how do I get this.
ReplyDeleteThanks, Jeff jdoty@flowcorp.com
Hi Jeff,
DeleteIt is also available out on my GitHub repo (oip file and all source code). Let me know if you have any issues.
https://github.com/jmattivi/Orchestrator/tree/master/Integration%20Packs/FTP-SFTP
Hi.
ReplyDeleteThis package works for orchestrator 2016?
Thanks
I can install the package, but the input properties don't appear
DeleteHi Francisco,
DeleteI have not personally tried the IP with 2016, but wouldn't foresee any reason why it shouldn't work. Previous comments have posted about using it with 2016 with no issues reported.
Could you try re-registering the IP and deploying to the designer?
Let me know,
Jon
Hi! We have a bunch of old but still functioning runbooks using ftp ip. Is there any possibility to have an updated version from FTP IP which would support the latest encryption methods? The provider is about to upgrade the sftp server and the current algorithms will not work after the upgrade. We must recreate the runbooks if the upgrade of your robust IP is not possible. Anyway, thanks for these years :)
ReplyDeleteHello,
DeleteCan you confirm if you're using SFTP for the current transfers? I don't believe there should be any impact from the client side if the provider is updating their sftp server. There shouldn't be any need to recreate the runbooks or find an alternate transfer method.
Jon
Hi!
DeleteWe tried again, but the encryption does not match:
Password authentication failed
Access denied
Sent password
Password authentication failed
Access denied
Disconnected: No supported authentication methods available (server sent: gssapi-with-mic,publickey)
Fatal: Disconnected: No supported authentication methods available (server sent: gssapi-with-mic,publickey)
Hello,
DeleteCan you repro the error or confirm with the latest version of psftp manually from the command line if that resolves the issue?
Jon
I did not test this with psftp, the transfer works fine with WinSCP. So are you saying that psftp should be updated? PSFTP is not installed on runbook servers, how do I update the version that FTP IP uses?
DeleteThe integration pack uses psftp (an older version at this point). Could you try using an updated version of psftp from the command line to see if the transfer is successful? If that is the case, it would appear the older version is no longer working and would need to be updated/repacked within the IP.
Delete