tech.v3.io

Wrapper for tech io subsystem designed to be drop in replacement for some use cases of clojure.java.io.

autodetect-csv-separator

(autodetect-csv-separator url & options)

Scan first 100 characters, count commas and tabs. Whichever one wins is the separator.

buffered-input-stream

(buffered-input-stream url & options)

copy

(copy src dest & args)

csv->mapseq

(csv->mapseq url & options)

Given a csv, produce a sequence of maps. This is mainly to be used for specific use cases like processing the data. For large datasets that you intend to do pandas-style dataset processing, please see tech.ml.dataset. If the input is not a reader, the sequence is completely read so the input stream can be closed. If the input is a reader, a lazy sequence of maps is returned.

The delimiter is auto-detected by scanning the first 100 or so characters. :separator - hard set csv separator to use. :skip-autodetect - Don’t autodetect, use ‘,’ as separator.

default-csv-key-printer

(default-csv-key-printer item-key)

delete!

(delete! url & options)

Delete a resource. Works currently with file or s3.

exists?

(exists? url & options)

Boolean existence check. Works with everything as fallback is to open an input stream and then close it.

file

(file path-or-url)

Wrapper around

get-edn

(get-edn url & options)

Get edn data from a url

get-image

(get-image url & options)

get-json

(get-json url & options)

Read json. Options are used both for constructing input stream and passed into the json read method. See documentation for clojure.data.json/read

get-nippy

(get-nippy url & options)

Get nippy data from a url

get-object

(get-object url & options)

Get object always returns something convertible to an input-stream. It may return a file for instance.

gzip-input-stream

(gzip-input-stream url & options)

gzip-output-stream!

(gzip-output-stream! url & options)

input-stream

(input-stream url & options)

thing->input-stream conversion. Falls back to clojure.java.io if url is not a string url

interlocked-copy-to-file

(interlocked-copy-to-file src dest & options)

Copy first to a temp, then do an atomic move to the destination. This avoids issues with partial files showing up where they shouldn’t and a failed io operation leading to incomplete results.

ls

(ls url & options)

Return a directory listing. May be recursive if desired; only works with file or s3 providers.

make-parents

mapseq->csv!

(mapseq->csv! url map-seq & options)

Given a sequence of maps, produce a csv or tsv. Options are passed via apply to clojure.data.csv/write-csv. Valid options for this method are: :key-printer - default print method to use else default-csv-key-printer :key-seq - ordered sequence of keys to pull out of the maps. Defaults to sorted order of printed keys of the first map. :separator - Which separator to use, defaults to ‘,’

metadata

(metadata url & options)

If supported, returns at least :modify-date and :byte-length. Exception otherwise.

output-stream!

(output-stream! url & options)

thing->output-stream conversion. Falls back to clojure.java.io if url is not a string url.

put-edn!

(put-edn! url data & options)

Put edn data to a url

put-image!

(put-image! url image & options)

Will throw if an image with transparency is used to write a jpeg

put-json!

(put-json! url data & options)

Write json. Options are used both for constructing the output stream and passed into the json write method. See documentation for clojure.data.json/write.

put-nippy!

(put-nippy! url data & options)

Put nippy data to a url

put-object!

(put-object! url value & options)

Put object. Object must be a byte-array, a file, or an input-stream. Strings will be interpreted as per the rules of clojure.java.io/input-stream. The most optimizations will apply to either files or byte arrays.

reader

(reader url & options)

Create a java.io.reader from a thing.

resource

(resource & args)

writer!

(writer! url & options)

Create a java.io.writer from a thing.