Consistent off by one errors in go io.copy function for large content -
this 1 works consistently.
    _, err = io.copy(out, resp.body)     if err != nil {         errlog.fatal(err)     } this 1 gives pretty consistent off 1 errors (the last byte of downloaded content left off, in case closing ] in json response) large-ish responses (mbs).
    if _, err := io.copy(out, resp.body); err != nil {         errlog.fatal(err)     } from the examples on official golang blog, looks should valid syntax.
edit: more details , context
this error 2nd version of code (more compact error handling)
error: 2015/08/05 08:09:31 pull.go:257: unexpected end of json input from code in function
err = json.unmarshal(dat, &all_data) if err != nil {     return err } i found off 1 issue looking @ first 10 , last 10 characters of file in each case. here before , afters:
# before (with error) start:  [{"tags":[              end:    ersion":1} start:  [{"_create              end:    "tags":[]}  # after start:  [{"tags":[              end:    rsion":1}] start:  [{"_create              end:    tags":[]}] the files 15-20 mb json strings.
it turned out issue @ least partially result of race condition.
i not calling .close() on file out before exiting function.  after adding have not been more issues.
why causing just last byte of file dropped mystery me.
Comments
Post a Comment