Aborting a GET in progress

[Jeff Johnson]

On Mar 18, 2010, at 4:50 AM, Joe Orton wrote:

> Hi Jeff!
> 
> On Wed, Mar 17, 2010 at 08:00:04PM -0400, Jeff Johnson wrote:
>> Is it feasible to abort a GET operation in progress but
>> preserve the connection session for performance?
> 
> To "abort" a request it's necessary to either read the entire response 
> body off the connection, or close it.
> 
>> Is it sufficient to destroy a request to abort an operation
>> or should I proceed to attempt ne_close_connection() or
>> even destroy/recreate the session in order to abort a
>> transfer in progress?
> 
> Destroying the request object then calling ne_close_connection() would 
> be sufficient.
> 

Thanks. That's what I needed to hear.

> I'm presuming you are using ne_begin_request()/ne_end_request(), and 
> that you have a good estimate of the total response length based on the 
> RPM header metadata from the response block(s) which are read?
> 

Yes.

> You have a trade-off: reading and discarding the entire response may be 
> slow if it's huge, but, if it's small then it may be faster than closing 
> the connection and re-opening.  I'd recommend some kind of heuristic:
> 
>  if expected-response-length < 256K (maybe):
>    ne_discard_response()
>    ne_end_request()
>    ne_request_destroy()
>  else:
>    ne_request_destroy()
>    close connection
> 

Ah nice touch. Lemme see if I can get some measurements for tuning
the "256K" heuristic. I'm about to be calling ne_close_connection()
frequently, putting a stopwatch on the abort won't be hard.

But it also sounds like its time for me to get off my duff and
do GET w byte ranges so that a persistent connection can 
be maintained. Draining a byte range is likley cheaper
than closing the connection (judging from yr "256K" heuristic).

(aside)
BTW, I mostly have my RPMIO API functional through neon these days,
linking into common unix tools from *BSD and doing s/fopen/Fopen/ within,
to permit URI's to be used everywhere that file paths are used.

E.g.
	/usr/lib/rpm/bin/find http://rpm5.org/files/ -name '*.rpm'
functions, and I have cp(1), grep(1), mtree(8) and (sufficiently) wget(1)
all tricked up to use Fopen(3) rather than fopen(3) on remote URI's.

Underneath the hood, your cadaver code to do "ls" has morph'ed to
Opendir(3) through a "argv dir" rather than a DAV collection. 

Here's the pattern I use from HTML parsing since plain HTML != a WebDAV collection in XML:

static const char * hrefpat =
    "(?i)<a(?:\\s+[a-z][a-z0-9_]*(?:=(?:\"[^\"]*\"|\\S+))?)*?\\s+href=(?:\"([^\"]*)\"|(\\S+))";

WORKS even if insane ;-)

I'm able to remotely tree walk large (>200K) *.rpm repositories
at gatech and duke .edu in 5-7 minutesusing Fts(3).

And all the fault of your cadaver code ;-)

73 de Jeff	off to hack in ne_close_connection()