Actinic upload system very poor

[pinbrook_nick]

I don't know what everyone else thinks, but I have always found the Actinic inbuilt upload system very poor in comparison with FTP clients, and other software with built in FTP (Frontpage, Dreamweaver etc). It can be very slow, very unreliable and unstable, and is highly unsophisticated (no scheduling / autoresume).

May I suggest that either the Actinic FTP coding receive a complete overhaul, or they do a deal with an external company who provide FTP software, such as Ipswitch (WS_FTP).

Features required for improved Actinic FTP coding:
1. Drastically improved upload speed.
2. More stable connections. (Actinic is always hanging and dropping connections, whereas FTP software to the same servers do not).
3. Auto-reconnect/resume of dropped transfers.
4. Scheduleable transfers.
5. "Quiet" mode uploads, i.e. a tick box to assure absolutely NO error messages appear. (You don't want to set an upload off before you go to bed, and come back to find an error message has appeared just before the whole upload started).

When considering why this is important, remember that there are a lot of people without the ability to have broadband, and a lot of people with shops exceeding 50MB. (50MB = nearly 5 hours on a dial-up).


Suggestions for collaboration with 3rd party:
As an example, WS_FTP from www.ipswitch.com has a scripting language, a scheduling tool, and command line control.
1. Have a feature in Actinic which can generate a script file for an external FTP program. The feature would have the option to generate code for an update or a refresh.
2. The FTP program could be launched straight away, or a scheduled transfer task created, either in the FTP programs' scheduler, or Windows Task Scheduler.
3. Actinic should have a feature which allows a user to click a button, and all files are marked as having been updated.


Comments? :rolleyes:

[Jan]

I agree, but would like to point out that for ======================
5. "Quiet" mode uploads, i.e. a tick box to assure absolutely NO error messages appear. (You don't want to set an upload off
before you go to bed, and come back to find an error message has appeared just before the whole upload started). ================

you can use my Automation Mole or my Exchange Rate Mole (details at http://www.mole-end.biz/acatalog/Actinic_Moles.html). You can set then up to automatically clear any dialogs that appear - so if Actiinic can upload with you sitting there it can upload with my automation moles.

Regards

[cdicken]

Nick

Thank you for your instructive and informative feedback. I will pass it all onto the development team and see if they have any initial comments.

[paulh69]

server logs show that actinic does not re-use ftp sessions, it looks like it starts a new ftp session for each file uploaded? logwatch reports around 700 open/close ftp sessions

scripts to tar/gzip files before uploading and extract on the server would be useful

fix for the error merging files if site has been generated but did not get uploaded for some reason - merge will fail the next time the site is uploaded unless diff files are deleted

[cdicken]

Your comments have provoked much discussion in the development team. The reason each file has its own FTP session is something to do with the fact that we need to set the permissions on each file as we upload it.

I could not fully follow the discussion, but there have been plenty of ideas as a result on how we can improve the upload.

Hopefully these ideas will make it into the next release.

[pinbrook_nick]

One quick addition then; often, the md000001.pl file causes an error on upload. I usually just delete the local version, and do another update, which causes Actinic to generate another copy.

[pinbrook]

And another thing, please check for the presence of a cgi-bin, if none present create one with correct permissions

[Infolink]

Chris,

Originally posted by cdicken
Your comments have provoked much discussion in the development team. The reason each file has its own FTP session is something to do with the fact that we need to set the permissions on each file as we upload it.


That is nonsense!
There is no reason why a script that CHMODs the files to the correct permissions cannot be compiled then run when the upload is complete!
Plenty of programs do it that way - and looking at the directory on the server just about all have the same permissions!
The only limitation that I can see is that you may need to identify the server O/S to make sure it works, but its not like there are many permutations, plus if it couldnt identify it the old method could be used.

zipping up all the files prior to an upload would reduce upload time dramatically.

Phil

[NormanRouxel]

That is nonsense!
There is no reason why a script that CHMODs the files to the correct permissions cannot be compiled then run when the upload is complete!

I'm not sure if waiting that long (especially for dial-up users) would be a good idea. It's a live site you're usually updating and you want the permissions right as soon as possible.

If I remember right the old V3 way was via HTTP and opened and then closed a script based session for every file. If I recall right V5 and onwards moved to FTP so that around 25 simultaneous FTP threads could be implemented. Perhaps somethings changed since then.

Norman

[Infolink]

Used with a combination of zipping the files, uploading them, unzipping and then CHMOD it should be fine since it would be as fast as the server could run them!

Phil

[cdicken]

Thanks again. I have fed this back.

[zmagyar]

Originally posted by cdicken
Thanks again. I have fed this back. ... and now I'm here. :)And another thing, please check for the presence of a cgi-bin, if none present create one with correct permissionsI'm afraid this can not be done. Consider that Catalog should run on a wide variety of servers. Probably your system configuration allows you to create whatever directories you want on the server but most of the systems do not allow this. That is nonsense!
There is no reason why a script that CHMODs the files to the correct permissions cannot be compiled then run when the upload is complete! There are several reasons why we do not follow this way. One of the reasons is well spotted by Norman. But there can also be permission problems. Catalog sets the permissions as the FTP user while the script would do this as the effective CGI user. It doesn't make difference on most of the systems but it does on a few. Our product is sold in a few 1000 copies and it should work on these systems. Several changes can be done on the upload system to improve its performance but most of them would make Catalog unusable on a few percent on the systems (what we can not afford I'm afraid). E.g. drastic improvement can be achieved by usingUsed with a combination of zipping the files, uploading them, unzipping and then CHMOD it should be fine since it would be as fast as the server could run them! But such a change would make Catalog unusable on a significant percent of the systems (unless we implement a pure perl zip library). The users of the unsupported systems wouldn't really enjoy the improved performance (would you?).

Otherwise I agree that there are some things which can be improved on our upload system. The original posting is recorded and we are going to look into the problem to make the upload system better.

Regards,

[pinbrook]

Another idea to help out peeps with large catalogs and slow dial-ups....can there be a refresh option with excludes images.

ie remove all html, pl, fil, cart, cat etc etc files, but leave all the images (gif, jpg and png) in place.

[pinbrook]

And another thing, please check for the presence of a cgi-bin, if none present create one with correct permissions

I'm afraid this can not be done. Consider that Catalog should run on a wide variety of servers. Probably your system configuration allows you to create whatever directories you want on the server but most of the systems do not allow this.

*E-R-O-L* creates a cgi-bin for you.....

There may be a wide variety of servers but doesn't it all boil down to UNIX or Windows

[Infolink]

yep and if you cant detect then it defaults back to the old way........

It could just fit into the network setting somewhere

Phil

[zmagyar]

*E-R-O-L* creates a cgi-bin for you........ and there are lot of things what it doesn't do but Actinic does. As I have already mentioned above I agree with you, there are lot of things which can be improved. And believe me, we are working hard to have as good product as possible. There may be a wide variety of servers but doesn't it all boil down to UNIX or WindowsThere is nothing to do with the OS. By "wide variety of servers" I meant server configuration rather than OS. There are configurations (e.g. CGI wrapper) where you can create a cgi-bin folder anywhere inside the web space and all scripts uploaded to this folder will be executed. But it is a very special case. Most of the systems use the web server configuration file to specify the location of the cgi-bin folder and execute scripts only from this folder. In this case you can create the cgi-bin folder anywhere (other than the location specified in the server config) the scripts in this folder will never be executed.
Generally the cgi-bin folder is owned by the web server, Catalog only uses this. Catalog only manipulates folders under its own full control.
I hope the above made this issue more clear.

Regards,

[Darren B]

i know this thread has gone on but i will leave the tech stuff to you guys,

but the most annoying, as mentioned ealier, is the GIF files these are always uploaded before the HTML pages. (i guess alphabetical) All the scripts are in place so we load GIF then HTML?

Just my 2 penith worth

Darren

[pinbrook]

Just thought I'd bump this, as there is another discussion about uploading issues.............is there anything in the pipline to improve uploading?

[cdicken]

Nothing planned right now. At the moment we want to concentrate on developing a solid range of features, before looking at issues of underlying technology.

In the longer term, we do want to provide alternatives for the underlying technological structure of Actinic - but that is a way off.