Consistent off by one errors in go io.copy function for large content -
this 1 works consistently.
_, err = io.copy(out, resp.body) if err != nil { errlog.fatal(err) }
this 1 gives pretty consistent off 1 errors (the last byte of downloaded content left off, in case closing ]
in json response) large-ish responses (mbs).
if _, err := io.copy(out, resp.body); err != nil { errlog.fatal(err) }
from the examples on official golang blog, looks should valid syntax.
edit: more details , context
this error 2nd version of code (more compact error handling)
error: 2015/08/05 08:09:31 pull.go:257: unexpected end of json input
from code in function
err = json.unmarshal(dat, &all_data) if err != nil { return err }
i found off 1 issue looking @ first 10 , last 10 characters of file in each case. here before , afters:
# before (with error) start: [{"tags":[ end: ersion":1} start: [{"_create end: "tags":[]} # after start: [{"tags":[ end: rsion":1}] start: [{"_create end: tags":[]}]
the files 15-20 mb json strings.
it turned out issue @ least partially result of race condition.
i not calling .close()
on file out
before exiting function. after adding have not been more issues.
why causing just last byte of file dropped mystery me.
Comments
Post a Comment