Consistent off by one errors in go io.copy function for large content -


this 1 works consistently.

    _, err = io.copy(out, resp.body)     if err != nil {         errlog.fatal(err)     } 

this 1 gives pretty consistent off 1 errors (the last byte of downloaded content left off, in case closing ] in json response) large-ish responses (mbs).

    if _, err := io.copy(out, resp.body); err != nil {         errlog.fatal(err)     } 

from the examples on official golang blog, looks should valid syntax.

edit: more details , context

this error 2nd version of code (more compact error handling)

error: 2015/08/05 08:09:31 pull.go:257: unexpected end of json input 

from code in function

err = json.unmarshal(dat, &all_data) if err != nil {     return err } 

i found off 1 issue looking @ first 10 , last 10 characters of file in each case. here before , afters:

# before (with error) start:  [{"tags":[              end:    ersion":1} start:  [{"_create              end:    "tags":[]}  # after start:  [{"tags":[              end:    rsion":1}] start:  [{"_create              end:    tags":[]}] 

the files 15-20 mb json strings.

it turned out issue @ least partially result of race condition.

i not calling .close() on file out before exiting function. after adding have not been more issues.

why causing just last byte of file dropped mystery me.


Comments

Popular posts from this blog

python - pip install -U PySide error -

arrays - C++ error: a brace-enclosed initializer is not allowed here before ‘{’ token -

apache - setting document root in antoher partition on ubuntu -