tech.v3.io
Wrapper for tech io subsystem designed to be drop in replacement for some use cases of clojure.java.io.
autodetect-csv-separator
(autodetect-csv-separator url & options)
Scan first 100 characters, count commas and tabs. Whichever one wins is the separator.
csv->mapseq
(csv->mapseq url & options)
Given a csv, produce a sequence of maps. This is mainly to be used for specific use cases like processing the data. For large datasets that you intend to do pandas-style dataset processing, please see tech.ml.dataset. If the input is not a reader, the sequence is completely read so the input stream can be closed. If the input is a reader, a lazy sequence of maps is returned.
The delimiter is auto-detected by scanning the first 100 or so characters. :separator - hard set csv separator to use. :skip-autodetect - Don’t autodetect, use ‘,’ as separator.
exists?
(exists? url & options)
Boolean existence check. Works with everything as fallback is to open an input stream and then close it.
get-json
(get-json url & options)
Read json. Options are used both for constructing input stream and passed into the json read method. See documentation for clojure.data.json/read
get-object
(get-object url & options)
Get object always returns something convertible to an input-stream. It may return a file for instance.
input-stream
(input-stream url & options)
thing->input-stream conversion. Falls back to clojure.java.io if url is not a string url
interlocked-copy-to-file
(interlocked-copy-to-file src dest & options)
Copy first to a temp, then do an atomic move to the destination. This avoids issues with partial files showing up where they shouldn’t and a failed io operation leading to incomplete results.
ls
(ls url & options)
Return a directory listing. May be recursive if desired; only works with file or s3 providers.
mapseq->csv!
(mapseq->csv! url map-seq & options)
Given a sequence of maps, produce a csv or tsv. Options are passed via apply to clojure.data.csv/write-csv. Valid options for this method are: :key-printer - default print method to use else default-csv-key-printer :key-seq - ordered sequence of keys to pull out of the maps. Defaults to sorted order of printed keys of the first map. :separator - Which separator to use, defaults to ‘,’
metadata
(metadata url & options)
If supported, returns at least :modify-date and :byte-length. Exception otherwise.
output-stream!
(output-stream! url & options)
thing->output-stream conversion. Falls back to clojure.java.io if url is not a string url.
put-image!
(put-image! url image & options)
Will throw if an image with transparency is used to write a jpeg
put-json!
(put-json! url data & options)
Write json. Options are used both for constructing the output stream and passed into the json write method. See documentation for clojure.data.json/write.
put-object!
(put-object! url value & options)
Put object. Object must be a byte-array, a file, or an input-stream. Strings will be interpreted as per the rules of clojure.java.io/input-stream. The most optimizations will apply to either files or byte arrays.