mirror of https://github.com/mpastell/Weave.jl
commit
11c720ad7f
|
@ -12,6 +12,7 @@ makedocs(
|
|||
"getting_started.md",
|
||||
"usage.md",
|
||||
"publish.md",
|
||||
"header.md",
|
||||
"chunk_options.md",
|
||||
"notebooks.md",
|
||||
"function_index.md",
|
||||
|
|
|
@ -8,6 +8,7 @@ Options are separated using ";" and need to be valid Julia expressions. Example:
|
|||
|
||||
Weave currently supports the following chunk options with the following defaults:
|
||||
|
||||
|
||||
## Options for code
|
||||
|
||||
- `echo = true`: Echo the code in the output document. If `false` the source code will be hidden.
|
||||
|
@ -21,6 +22,7 @@ Weave currently supports the following chunk options with the following defaults
|
|||
- `hold = false`: Hold all results until the end of the chunk.
|
||||
- `tangle = true`: Set tangle to `false` to exclude chunk from tangled code.
|
||||
|
||||
|
||||
## Options for figures
|
||||
|
||||
- `fig_width = 6`: Figure width passed to plotting library.
|
||||
|
@ -42,7 +44,7 @@ You can set the default chunk options (and `weave` arguments) for a document usi
|
|||
```yaml
|
||||
---
|
||||
options:
|
||||
out_width : 50%
|
||||
out_width : 50%
|
||||
---
|
||||
```
|
||||
|
||||
|
|
|
@ -0,0 +1,88 @@
|
|||
# Header Configuration
|
||||
|
||||
When `weave`ing a markdown document, you use YAML header to provide additional metadata and configuration options.
|
||||
A YAML header should be in the beginning of the input document delimited with `---`.
|
||||
|
||||
|
||||
!!! warning
|
||||
YAML header configuration is only supported when `weave`ing [Julia markdown documents](@ref document-syntax).
|
||||
|
||||
|
||||
## Document Metadata
|
||||
|
||||
You can set additional document metadata in YAML header.
|
||||
When `weave`ing to Julia markdown documents to HTML or PDF, Weave respects the following metadata specification:
|
||||
- `title`
|
||||
- `author`
|
||||
- `date`
|
||||
|
||||
An example:
|
||||
```yaml
|
||||
---
|
||||
title : Header Example
|
||||
author : Shuhei Kadowaki
|
||||
date: 16th May 2020
|
||||
---
|
||||
```
|
||||
|
||||
!!! note
|
||||
You can also have other metadata, but they won't appear in the resulting HTML and PDF.
|
||||
If you weave to Julia markdown to GitHub/Hugo markdown, all the metadata will be preserved.
|
||||
|
||||
### Dynamic Metadata
|
||||
|
||||
The metadata can be "dynamic"; if you have [inline code](@ref) within YAML header, they will be evaluated _after_ evaluating all the chunks and replaced with the results.
|
||||
|
||||
The example document below will set `date` metadata dynamically.
|
||||
Note that `Date` is available since the chunk is evaluated first.
|
||||
```md
|
||||
---
|
||||
title : Header Example
|
||||
author : Shuhei Kadowaki
|
||||
date: `j Date(now())`
|
||||
---
|
||||
|
||||
```julia; echo = false
|
||||
using Datas
|
||||
```
|
||||
```
|
||||
|
||||
|
||||
## Configuration Options
|
||||
|
||||
Each of keyword arguments of [`weave`](@ref) can be set in the YAML header under `options` field.
|
||||
You can also set [Chunks Options](@ref) there that will be applied globally.
|
||||
|
||||
The example below sets `out_path` and `doctype` options and overwrites `term` and `wrap` chunk options:
|
||||
```yaml
|
||||
---
|
||||
title : Header Example
|
||||
author : Shuhei Kadowaki
|
||||
date: 16th May 2020
|
||||
options:
|
||||
out_path: relative/path/to/this/document
|
||||
doctype: github
|
||||
term: true
|
||||
wrap: false
|
||||
---
|
||||
```
|
||||
|
||||
!!! note
|
||||
- configurations specified within the YAML header have higher precedence than those specified via `weave` keyword arguments
|
||||
- chunk options specified within each chunk have higher precedence than the global global chunk options specified within the YAML header
|
||||
|
||||
|
||||
## Format Specific Options
|
||||
|
||||
The header configurations can be format specific.
|
||||
Here is how to set different `out_path` for `md2html` and `md2pdf` and set `fig_ext` globally:
|
||||
```yaml
|
||||
---
|
||||
options:
|
||||
md2html:
|
||||
out_path : html
|
||||
md2pdf:
|
||||
out_path : pdf
|
||||
fig_ext : .png
|
||||
---
|
||||
```
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
# Weave.jl - Scientific Reports Using Julia
|
||||
|
||||
This is the documentation of [Weave.jl](http://github.com/mpastell/weave.jl).
|
||||
|
@ -22,10 +21,18 @@ and [Sweave](https://stat.ethz.ch/R-manual/R-patched/library/utils/doc/Sweave.pd
|
|||
|
||||
![Weave in Juno demo](https://user-images.githubusercontent.com/40514306/76081328-32f41900-5fec-11ea-958a-375f77f642a2.png)
|
||||
|
||||
## Contents
|
||||
|
||||
## Index
|
||||
|
||||
```@contents
|
||||
Pages = ["getting_started.md", "usage.md",
|
||||
"publish.md", "chunk_options.md", "notebooks.md",
|
||||
"function_index.md"]
|
||||
Pages = [
|
||||
"index.md",
|
||||
"getting_started.md",
|
||||
"usage.md",
|
||||
"publish.md",
|
||||
"header.md",
|
||||
"chunk_options.md",
|
||||
"notebooks.md",
|
||||
"function_index.md",
|
||||
]
|
||||
```
|
||||
|
|
|
@ -3,7 +3,7 @@
|
|||
You can write your documentation and code in input document using Markdown, Noweb or script
|
||||
syntax and use [`weave`](@ref) function to execute to document to capture results and figures.
|
||||
|
||||
## Weave
|
||||
## `weave`
|
||||
|
||||
Weave document with markup and julia code using `Plots.jl` for plots,
|
||||
`out_path = :pwd` makes the results appear in the current working directory.
|
||||
|
@ -21,7 +21,7 @@ weave(joinpath(dirname(pathof(Weave)), "../examples", "FIR_design.jmd"), out_pat
|
|||
weave
|
||||
```
|
||||
|
||||
## Tangle
|
||||
## `tangle`
|
||||
|
||||
Tangling extracts the code from document:
|
||||
|
||||
|
@ -29,7 +29,7 @@ Tangling extracts the code from document:
|
|||
tangle
|
||||
```
|
||||
|
||||
## Supported output formats
|
||||
## Supported Output Formats
|
||||
|
||||
Weave automatically detects the output format based on the file extension.
|
||||
The auto output format detection is handled by `detect_doctype(path::AbstractString)`:
|
||||
|
@ -59,7 +59,7 @@ using Weave # hide
|
|||
list_out_formats()
|
||||
```
|
||||
|
||||
## Document syntax
|
||||
## [Document Syntax](@id document-syntax)
|
||||
|
||||
Weave uses markdown, Noweb or script syntax for defining the code chunks and
|
||||
documentation chunks. You can also weave Jupyter notebooks. The format is detected based on the file extension, but you can also set it manually using the `informat` parameter.
|
||||
|
@ -73,13 +73,13 @@ ext == ".ipynb" && return "notebook"
|
|||
return "noweb"
|
||||
```
|
||||
|
||||
## Documentation chunks
|
||||
## Documentation Chunks
|
||||
|
||||
In Markdown and Noweb input formats documentation chunks are the parts that aren't inside code delimiters. Documentation chunks can be written with several different markup languages.
|
||||
|
||||
## Code chunks
|
||||
## Code Chunks
|
||||
|
||||
### Markdown format
|
||||
### Markdown Format
|
||||
|
||||
Markdown code chunks are defined using fenced code blocks with options following on the same line. e.g. to hide code from output you can use:
|
||||
|
||||
|
@ -89,7 +89,7 @@ Markdown code chunks are defined using fenced code blocks with options following
|
|||
|
||||
[Sample document]( https://github.com/mpastell/Weave.jl/blob/master/examples/FIR_design.jmd)
|
||||
|
||||
## Inline code
|
||||
## [Inline Code](@id inline-code)
|
||||
|
||||
You can also add inline code to your documents using
|
||||
|
||||
|
@ -121,60 +121,38 @@ or to produce any html output:
|
|||
```
|
||||
|
||||
|
||||
### Noweb format
|
||||
### Noweb Format
|
||||
|
||||
Code chunks start with a line marked with `<<>>=` or `<<options>>=` and end with line marked with `@`. The code between the start and end markers is executed and the output is captured to the output document. See [chunk options](../chunk_options/).
|
||||
|
||||
|
||||
### Script format
|
||||
### Script Format
|
||||
|
||||
Weave also support script input format with a markup in comments.
|
||||
These scripts can be executed normally using Julia or published with
|
||||
Weave. Documentation is in lines starting with
|
||||
`#'`, `#%%` or `# %%`, and code is executed and results are included
|
||||
in the weaved document.
|
||||
These scripts can be executed normally using Julia or published with Weave.
|
||||
|
||||
All lines that are not documentation are treated as code. You can set chunk options
|
||||
using lines starting with `#+` just before code e.g. `#+ term=true`.
|
||||
Lines starting with `#'`, `#%%` or `# %%` are treated as document.
|
||||
|
||||
The format is identical to [Pweave](http://mpastell.com/pweave/pypublish.html)
|
||||
and the concept is similar to publishing documents with MATLAB or
|
||||
using Knitr's [spin](http://yihui.name/knitr/demo/stitch/).
|
||||
All non-document lines are treated as code.
|
||||
You can set chunk options using lines starting with `#+` just before code e.g:
|
||||
```julia
|
||||
#+ term=true
|
||||
hoge # some code comes here
|
||||
```
|
||||
|
||||
The format is identical to [Pweave](http://mpastell.com/pweave/pypublish.html) and the concept is similar to publishing documents with MATLAB or using Knitr's [spin](http://yihui.name/knitr/demo/stitch/).
|
||||
Weave will remove the first empty space from each line of documentation.
|
||||
|
||||
|
||||
[See sample document:](https://github.com/mpastell/Weave.jl/blob/master/examples/FIR_design.jl)
|
||||
|
||||
## Setting document options in header
|
||||
|
||||
You can use a YAML header in the beginning of the input document delimited with "---" to set the document title, author and date e.g. and default document options. Each of Weave command line arguments and chunk options can be set in header using `options` field. Below is an example that sets document `out_path` and `doctype` using the header.
|
||||
## Configuration via YAML Header
|
||||
|
||||
When `weave`ing markdown files, you use YAML header to provide additional metadata and configuration options.
|
||||
See [Header Configuration](@ref) section for more details.
|
||||
|
||||
|
||||
```yaml
|
||||
---
|
||||
title : Weave example
|
||||
author : Matti Pastell
|
||||
date: 15th December 2016
|
||||
options:
|
||||
out_path : reports/example.md
|
||||
doctype : github
|
||||
---
|
||||
```
|
||||
|
||||
You can also set format specific options. Here is how to set different `out_path` for `md2html` and `md2pdf` and set `fig_ext` for both:
|
||||
|
||||
```
|
||||
---
|
||||
options:
|
||||
md2html:
|
||||
out_path : html
|
||||
md2pdf:
|
||||
out_path : pdf
|
||||
fig_ext : .png
|
||||
---
|
||||
```
|
||||
|
||||
## Passing arguments to documents
|
||||
## Passing Runtime Arguments to Documents
|
||||
|
||||
You can pass arguments as `Dict` to the weaved document using the `args` argument
|
||||
to `weave`. The arguments will be available as `WEAVE_ARGS` variable in the document.
|
||||
|
@ -192,18 +170,18 @@ weave("mydoc.jmd", args = Dict("filename" => "somedata.h5"))
|
|||
and you can access the filename from document as follows:
|
||||
|
||||
```
|
||||
```julia
|
||||
print(WEAVE_ARGS["filename"])
|
||||
```
|
||||
```julia
|
||||
print(WEAVE_ARGS["filename"])
|
||||
```
|
||||
```
|
||||
|
||||
You can use the `out_path` argument to control the name of the
|
||||
output document.
|
||||
|
||||
## Include Weave document in Julia
|
||||
|
||||
You can call `include_weave` on a Weave document to run the contents
|
||||
of all code chunks in Julia.
|
||||
## `include_weave`
|
||||
|
||||
You can call `include_weave` on a Weave document and run all code chunks within in the current session.
|
||||
|
||||
```@docs
|
||||
include_weave
|
||||
|
|
17
src/Weave.jl
17
src/Weave.jl
|
@ -3,6 +3,8 @@ module Weave
|
|||
using Highlights, Mustache, Requires
|
||||
|
||||
|
||||
const PKG_DIR = normpath(@__DIR__, "..")
|
||||
const TEMPLATE_DIR = normpath(PKG_DIR, "templates")
|
||||
const WEAVE_OPTION_NAME = "options" # TODO: rename to "weave_options"
|
||||
|
||||
function __init__()
|
||||
|
@ -114,8 +116,7 @@ function weave(
|
|||
latex_cmd::AbstractString = "xelatex",
|
||||
latex_keep_unicode::Bool = false,
|
||||
)
|
||||
doc = WeaveDoc(source, informat)
|
||||
isnothing(doctype) && (doctype = detect_doctype(doc.source))
|
||||
doc = WeaveDoc(source, informat, doctype)
|
||||
|
||||
# overwrites given options with header options, which have more precedence
|
||||
# NOTE:
|
||||
|
@ -124,7 +125,7 @@ function weave(
|
|||
weave_options = get(doc.header, WEAVE_OPTION_NAME, Dict())
|
||||
if !isempty(weave_options)
|
||||
doctype = get(weave_options, "doctype", doctype)
|
||||
weave_options = combine_args(weave_options, doctype)
|
||||
specific_options!(weave_options, doctype)
|
||||
if haskey(weave_options, "out_path")
|
||||
out_path = let
|
||||
out_path = weave_options["out_path"]
|
||||
|
@ -201,6 +202,16 @@ function weave(
|
|||
return abspath(outname)
|
||||
end
|
||||
|
||||
function specific_options!(weave_options, doctype)
|
||||
fmts = keys(formats)
|
||||
for (k,v) in weave_options
|
||||
if k in fmts
|
||||
k == doctype && merge!(weave_options, v)
|
||||
delete!(weave_options, k)
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
weave(doc::AbstractString, doctype::Union{Symbol,AbstractString}) =
|
||||
weave(doc; doctype = doctype)
|
||||
|
||||
|
|
|
@ -62,18 +62,3 @@ get_chunk_defaults() = rcParams[:chunk_defaults]
|
|||
Restore Weave.jl default chunk options.
|
||||
"""
|
||||
restore_chunk_defaults!() = rcParams[:chunk_defaults] = defaultParams[:chunk_defaults]
|
||||
|
||||
"""Combine format specific and common options from document header"""
|
||||
function combine_args(args, doctype)
|
||||
common = Dict()
|
||||
specific = Dict()
|
||||
for key in keys(args)
|
||||
if key in keys(Weave.formats)
|
||||
specific[key] = args[key]
|
||||
else
|
||||
common[key] = args[key]
|
||||
end
|
||||
end
|
||||
haskey(specific, doctype) && merge!(common, specific[doctype])
|
||||
common
|
||||
end
|
||||
|
|
|
@ -92,19 +92,6 @@ function convert_to_notebook(doc)
|
|||
cells = []
|
||||
ex_count = 1
|
||||
|
||||
# Handle header
|
||||
head_tpl = """
|
||||
{{#:title}}# {{:title}}{{/:title}}
|
||||
{{#:author}}### {{{:author}}}{{/:author}}
|
||||
{{#:date}}### {{{:date}}}{{/:date}}
|
||||
"""
|
||||
|
||||
if isa(doc.chunks[1], DocChunk)
|
||||
strip_header!(doc)
|
||||
doc.chunks[1].content[1].content =
|
||||
Mustache.render(head_tpl; [Pair(Symbol(k), v) for (k, v) in doc.header]...) * doc.chunks[1].content[1].content
|
||||
end
|
||||
|
||||
for chunk in doc.chunks
|
||||
if isa(chunk, DocChunk)
|
||||
push!(
|
||||
|
|
111
src/format.jl
111
src/format.jl
|
@ -1,5 +1,4 @@
|
|||
using Mustache, Highlights
|
||||
using .WeaveMarkdown, Markdown, Dates
|
||||
using Mustache, Highlights, .WeaveMarkdown, Markdown, Dates
|
||||
using REPL.REPLCompletions: latex_symbols
|
||||
|
||||
function format(doc::WeaveDoc)
|
||||
|
@ -18,7 +17,7 @@ function format(doc::WeaveDoc)
|
|||
docformat.formatdict[:cwd] = doc.cwd # pass wd to figure formatters
|
||||
docformat.formatdict[:theme] = doc.highlight_theme
|
||||
|
||||
strip_header!(doc)
|
||||
restore_header!(doc)
|
||||
|
||||
for chunk in copy(doc.chunks)
|
||||
result = format_chunk(chunk, formatdict, docformat)
|
||||
|
@ -36,11 +35,11 @@ render_doc(formatted, doc, format) = formatted
|
|||
|
||||
function highlight(
|
||||
mime::MIME,
|
||||
source::AbstractString,
|
||||
output,
|
||||
lexer,
|
||||
theme = Highlights.Themes.DefaultTheme,
|
||||
)
|
||||
return sprint((io, x) -> Highlights.highlight(io, mime, x, lexer, theme), source)
|
||||
return sprint((io, x) -> Highlights.highlight(io, mime, x, lexer, theme), output)
|
||||
end
|
||||
|
||||
function stylesheet(m::MIME, theme)
|
||||
|
@ -54,22 +53,14 @@ function render_doc(formatted, doc, format::JMarkdown2HTML)
|
|||
wversion = ""
|
||||
wtime = string(Date(now()))
|
||||
|
||||
if isempty(doc.css)
|
||||
theme_css =
|
||||
read(joinpath(dirname(@__FILE__), "../templates/skeleton_css.css"), String)
|
||||
else
|
||||
theme_css = read(doc.css, String)
|
||||
end
|
||||
theme_path = isempty(doc.css) ? normpath(TEMPLATE_DIR, "skeleton_css.css") : doc.css
|
||||
theme_css = read(theme_path, String)
|
||||
|
||||
if isa(doc.template, Mustache.MustacheTokens)
|
||||
template = doc.template
|
||||
elseif isempty(doc.template)
|
||||
template = Mustache.template_from_file(joinpath(
|
||||
dirname(@__FILE__),
|
||||
"../templates/julia_html.tpl",
|
||||
))
|
||||
template = if isa(doc.template, Mustache.MustacheTokens)
|
||||
doc.template
|
||||
else
|
||||
template = Mustache.template_from_file(doc.template)
|
||||
template_path = isempty(doc.template) ? normpath(TEMPLATE_DIR, "julia_html.tpl") : doc.template
|
||||
Mustache.template_from_file(template_path)
|
||||
end
|
||||
|
||||
return Mustache.render(
|
||||
|
@ -92,15 +83,11 @@ function render_doc(formatted, doc, format::JMarkdown2tex)
|
|||
wversion = ""
|
||||
wtime = string(Date(now()))
|
||||
|
||||
if isa(doc.template, Mustache.MustacheTokens)
|
||||
template = doc.template
|
||||
elseif isempty(doc.template)
|
||||
template = Mustache.template_from_file(joinpath(
|
||||
dirname(@__FILE__),
|
||||
"../templates/julia_tex.tpl",
|
||||
))
|
||||
template = if isa(doc.template, Mustache.MustacheTokens)
|
||||
doc.template
|
||||
else
|
||||
template = Mustache.template_from_file(doc.template)
|
||||
template_path = isempty(doc.template) ? normpath(TEMPLATE_DIR, "julia_tex.tpl") : doc.template
|
||||
Mustache.template_from_file(template_path)
|
||||
end
|
||||
|
||||
return Mustache.render(
|
||||
|
@ -111,40 +98,27 @@ function render_doc(formatted, doc, format::JMarkdown2tex)
|
|||
)
|
||||
end
|
||||
|
||||
strip_header!(doc::WeaveDoc) = strip_header!(doc.chunks[1], doc.doctype)
|
||||
function strip_header!(docchunk::DocChunk, doctype)
|
||||
doctype == "pandoc" && return
|
||||
content = docchunk.content[1].content
|
||||
if (m = match(HEADER_REGEX, content)) !== nothing
|
||||
# TODO: is there other format where we want to keep headers ?
|
||||
docchunk.content[1].content = if doctype != "github"
|
||||
lstrip(replace(content, HEADER_REGEX => ""))
|
||||
else
|
||||
# only strips Weave headers
|
||||
header = YAML.load(m[:header])
|
||||
delete!(header, WEAVE_OPTION_NAME)
|
||||
if isempty(header)
|
||||
lstrip(replace(content, HEADER_REGEX => ""))
|
||||
else
|
||||
lstrip(replace(content, HEADER_REGEX => "---\n$(YAML.write(header))---"))
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
strip_header!(codechunk::CodeChunk, doctype) = return
|
||||
function restore_header!(doc)
|
||||
# TODO: is there any other format where we want to restore headers ?
|
||||
doc.doctype ≠ "github" && return
|
||||
|
||||
function format_chunk(chunk::DocChunk, formatdict, docformat)
|
||||
return join([format_inline(c) for c in chunk.content], "")
|
||||
# only strips Weave headers
|
||||
delete!(doc.header, WEAVE_OPTION_NAME)
|
||||
isempty(doc.header) && return
|
||||
|
||||
# restore remained headers as `DocChunk`
|
||||
header_text = "---\n$(YAML.write(doc.header))---"
|
||||
pushfirst!(doc.chunks, DocChunk(header_text, 0, 0))
|
||||
end
|
||||
|
||||
function format_inline(inline::InlineText)
|
||||
return inline.content
|
||||
end
|
||||
format_chunk(chunk::DocChunk, formatdict, docformat) = join((format_inline(c) for c in chunk.content))
|
||||
|
||||
format_inline(inline::InlineText) = inline.content
|
||||
|
||||
function format_inline(inline::InlineCode)
|
||||
isempty(inline.rich_output) || return inline.rich_output
|
||||
isempty(inline.figures) || return inline.figures[end]
|
||||
isempty(inline.output) || return inline.output
|
||||
return inline.output
|
||||
end
|
||||
|
||||
function ioformat!(io::IOBuffer, out::IOBuffer, fun = WeaveMarkdown.latex)
|
||||
|
@ -155,10 +129,7 @@ function ioformat!(io::IOBuffer, out::IOBuffer, fun = WeaveMarkdown.latex)
|
|||
end
|
||||
end
|
||||
|
||||
function addspace(op, inline)
|
||||
inline.ctype == :line && (op = "\n$op\n")
|
||||
return op
|
||||
end
|
||||
addspace(op, inline) = (inline.ctype === :line && (op = "\n$op\n"); op)
|
||||
|
||||
function format_chunk(chunk::DocChunk, formatdict, docformat::JMarkdown2tex)
|
||||
out = IOBuffer()
|
||||
|
@ -275,15 +246,11 @@ function format_chunk(chunk::CodeChunk, formatdict, docformat)
|
|||
return result
|
||||
end
|
||||
|
||||
function format_output(result::AbstractString, docformat)
|
||||
return result
|
||||
end
|
||||
format_output(result, docformat) = result
|
||||
|
||||
function format_output(result::AbstractString, docformat::JMarkdown2HTML)
|
||||
return Markdown.htmlesc(result)
|
||||
end
|
||||
format_output(result, docformat::JMarkdown2HTML) = Markdown.htmlesc(result)
|
||||
|
||||
function format_output(result::AbstractString, docformat::JMarkdown2tex)
|
||||
function format_output(result, docformat::JMarkdown2tex)
|
||||
# Highligts has some extra escaping defined, eg of $, ", ...
|
||||
result_escaped = sprint(
|
||||
(io, x) ->
|
||||
|
@ -294,11 +261,9 @@ function format_output(result::AbstractString, docformat::JMarkdown2tex)
|
|||
return result_escaped
|
||||
end
|
||||
|
||||
function format_code(result::AbstractString, docformat)
|
||||
return result
|
||||
end
|
||||
format_code(result, docformat) = result
|
||||
|
||||
function format_code(result::AbstractString, docformat::JMarkdown2tex)
|
||||
function format_code(result, docformat::JMarkdown2tex)
|
||||
highlighted = highlight(
|
||||
MIME("text/latex"),
|
||||
strip(result),
|
||||
|
@ -339,7 +304,7 @@ function texify(s)
|
|||
return ts
|
||||
end
|
||||
|
||||
function format_code(result::AbstractString, docformat::JMarkdown2HTML)
|
||||
function format_code(result, docformat::JMarkdown2HTML)
|
||||
return highlight(
|
||||
MIME("text/html"),
|
||||
strip(result),
|
||||
|
@ -348,7 +313,7 @@ function format_code(result::AbstractString, docformat::JMarkdown2HTML)
|
|||
)
|
||||
end
|
||||
|
||||
function format_code(result::AbstractString, docformat::Pandoc2HTML)
|
||||
function format_code(result, docformat::Pandoc2HTML)
|
||||
return highlight(
|
||||
MIME("text/html"),
|
||||
strip(result),
|
||||
|
@ -409,9 +374,7 @@ function format_termchunk(chunk, formatdict, docformat::JMarkdown2tex)
|
|||
return result
|
||||
end
|
||||
|
||||
function indent(text, nindent)
|
||||
return join(map(x -> string(repeat(" ", nindent), x), split(text, "\n")), "\n")
|
||||
end
|
||||
indent(text, nindent) = join(map(x -> string(repeat(' ', nindent), x), split(text, '\n')), '\n')
|
||||
|
||||
function wraplines(text, line_width = 75)
|
||||
result = AbstractString[]
|
||||
|
|
|
@ -509,7 +509,7 @@ function formatfigures(chunk, docformat::AsciiDoc)
|
|||
end
|
||||
|
||||
# Add new supported formats here
|
||||
const formats = Dict{AbstractString,Any}(
|
||||
const formats = Dict(
|
||||
"tex" => tex,
|
||||
"texminted" => texminted,
|
||||
"pandoc" => pandoc,
|
||||
|
|
|
@ -1,6 +1,5 @@
|
|||
|
||||
"""
|
||||
`pandoc2html(formatted::AbstractString, doc::WeaveDoc)`
|
||||
pandoc2html(formatted::AbstractString, doc::WeaveDoc)
|
||||
|
||||
Convert output from pandoc markdown to html using Weave.jl template
|
||||
"""
|
||||
|
@ -38,10 +37,6 @@ function pandoc2html(
|
|||
html = ""
|
||||
outname = basename(outname)
|
||||
|
||||
open("temp.md", "w") do io
|
||||
println(io, formatted)
|
||||
end
|
||||
|
||||
try
|
||||
cmd = `pandoc -f markdown+raw_html -s --mathjax=""
|
||||
$filt $citeproc $pandoc_options
|
||||
|
@ -63,7 +58,7 @@ function pandoc2html(
|
|||
end
|
||||
|
||||
"""
|
||||
`pandoc2pdf(formatted::AbstractString, doc::WeaveDoc)`
|
||||
pandoc2pdf(formatted::AbstractString, doc::WeaveDoc)
|
||||
|
||||
Convert output from pandoc markdown to pdf using Weave.jl template
|
||||
"""
|
||||
|
@ -95,7 +90,7 @@ function pandoc2pdf(
|
|||
citeproc = []
|
||||
end
|
||||
|
||||
@info("Done executing code. Running xelatex")
|
||||
@info "Done executing code. Running xelatex"
|
||||
try
|
||||
cmd = `pandoc -f markdown+raw_tex -s --pdf-engine=xelatex --highlight-style=tango
|
||||
$filt $citeproc $pandoc_options
|
||||
|
@ -117,7 +112,7 @@ function run_latex(doc::WeaveDoc, outname, latex_cmd = "xelatex")
|
|||
old_wd = pwd()
|
||||
cd(doc.cwd)
|
||||
xname = basename(outname)
|
||||
@info("Weaved code to $outname . Running $latex_cmd") # space before '.' added for link to be clickable in Juno terminal
|
||||
@info "Weaved code to $outname . Running $latex_cmd" # space before '.' added for link to be clickable in Juno terminal
|
||||
textmp = mktempdir(".")
|
||||
try
|
||||
out = read(
|
||||
|
|
|
@ -1,19 +1,16 @@
|
|||
using JSON, YAML
|
||||
|
||||
|
||||
function WeaveDoc(source, format::Union{Nothing,AbstractString} = nothing)
|
||||
document = replace(read(source, String), "\r\n" => "\n") # normalize line ending
|
||||
isnothing(format) && (format = detect_informat(source))
|
||||
chunks = parse_doc(document, format)
|
||||
return WeaveDoc(source, chunks)
|
||||
end
|
||||
|
||||
function WeaveDoc(source, chunks::Vector{WeaveChunk})
|
||||
function WeaveDoc(source, informat = nothing, doctype = nothing)
|
||||
path, fname = splitdir(abspath(source))
|
||||
basename = splitext(fname)[1]
|
||||
|
||||
header = parse_header(first(chunks))
|
||||
# get chunk defaults from header and update
|
||||
isnothing(informat) && (informat = detect_informat(source))
|
||||
header, chunks = parse_doc(read(source, String), informat)
|
||||
|
||||
isnothing(doctype) && (doctype = detect_doctype(source))
|
||||
|
||||
# update default chunk options from header
|
||||
chunk_defaults = deepcopy(get_chunk_defaults())
|
||||
if haskey(header, WEAVE_OPTION_NAME)
|
||||
for key in keys(chunk_defaults)
|
||||
|
@ -30,7 +27,7 @@ function WeaveDoc(source, chunks::Vector{WeaveChunk})
|
|||
chunks,
|
||||
"",
|
||||
nothing,
|
||||
"",
|
||||
doctype,
|
||||
"",
|
||||
header,
|
||||
"",
|
||||
|
@ -42,12 +39,12 @@ function WeaveDoc(source, chunks::Vector{WeaveChunk})
|
|||
end
|
||||
|
||||
"""
|
||||
detect_informat(source::AbstractString)
|
||||
detect_informat(path)
|
||||
|
||||
Detect Weave input format based on file extension of `source`.
|
||||
Detect Weave input format based on file extension of `path`.
|
||||
"""
|
||||
function detect_informat(source::AbstractString)
|
||||
ext = lowercase(last(splitext(source)))
|
||||
function detect_informat(path)
|
||||
ext = lowercase(last(splitext(path)))
|
||||
|
||||
ext == ".jl" && return "script"
|
||||
ext == ".jmd" && return "markdown"
|
||||
|
@ -55,18 +52,17 @@ function detect_informat(source::AbstractString)
|
|||
return "noweb"
|
||||
end
|
||||
|
||||
function parse_doc(document, format)::Vector{WeaveChunk}
|
||||
return if format == "markdown"
|
||||
parse_markdown(document)
|
||||
elseif format == "noweb"
|
||||
parse_markdown(document, true)
|
||||
elseif format == "script"
|
||||
parse_script(document)
|
||||
elseif format == "notebook"
|
||||
parse_notebook(document)
|
||||
else
|
||||
error("unsupported format given: $(format)")
|
||||
end
|
||||
function parse_doc(document, informat)
|
||||
document = replace(document, "\r\n" => "\n") # normalize line ending
|
||||
|
||||
header_text, document = separate_header_text(document)
|
||||
|
||||
return parse_header(header_text),
|
||||
informat == "markdown" ? parse_markdown(document) :
|
||||
informat == "noweb" ? parse_markdown(document, true) :
|
||||
informat == "script" ? parse_script(document) :
|
||||
informat == "notebook" ? parse_notebook(document) :
|
||||
error("unsupported input format given: $informat")
|
||||
end
|
||||
|
||||
function pushopt(options::Dict, expr::Expr)
|
||||
|
@ -75,6 +71,22 @@ function pushopt(options::Dict, expr::Expr)
|
|||
end
|
||||
end
|
||||
|
||||
"""
|
||||
detect_doctype(path)
|
||||
|
||||
Detect the output format based on file extension.
|
||||
"""
|
||||
function detect_doctype(path)
|
||||
_, ext = lowercase.(splitext(path))
|
||||
|
||||
match(r"^\.(jl|.?md|ipynb)", ext) !== nothing && return "md2html"
|
||||
ext == ".rst" && return "rst"
|
||||
ext == ".tex" && return "texminted"
|
||||
ext == ".txt" && return "asciidoc"
|
||||
|
||||
return "pandoc"
|
||||
end
|
||||
|
||||
# inline
|
||||
# ------
|
||||
|
||||
|
@ -84,12 +96,13 @@ function DocChunk(text::AbstractString, number, start_line; notebook = false)
|
|||
return DocChunk(content, number, start_line)
|
||||
end
|
||||
|
||||
const INLINE_REGEX = r"`j\s+(.*?)`|^!\s(.*)$"m
|
||||
const INLINE_REGEX = r"`j\s+(.*?)`"
|
||||
const INLINE_REGEXES = r"`j\s+(.*?)`|^!\s(.*)$"m
|
||||
|
||||
function parse_inlines(text)::Vector{Inline}
|
||||
occursin(INLINE_REGEX, text) || return parse_inline(text)
|
||||
occursin(INLINE_REGEXES, text) || return parse_inline(text)
|
||||
|
||||
inline_chunks = eachmatch(INLINE_REGEX, text)
|
||||
inline_chunks = eachmatch(INLINE_REGEXES, text)
|
||||
s = 1
|
||||
e = 1
|
||||
res = Inline[]
|
||||
|
@ -118,20 +131,28 @@ parse_inline(text) = Inline[InlineText(text, 1, length(text), 1)]
|
|||
# headers
|
||||
# -------
|
||||
|
||||
parse_header(chunk::CodeChunk) = Dict()
|
||||
|
||||
const HEADER_REGEX = r"^---$(?<header>((?!---).)+)^---$"ms
|
||||
|
||||
function parse_header(chunk::DocChunk)
|
||||
m = match(HEADER_REGEX, chunk.content[1].content)
|
||||
if m !== nothing
|
||||
header = YAML.load(string(m[:header]))
|
||||
else
|
||||
header = Dict()
|
||||
end
|
||||
return header
|
||||
# TODO: non-Weave headers should keep live in a doc
|
||||
# separates header section from `text`
|
||||
function separate_header_text(text)
|
||||
m = match(HEADER_REGEX, text)
|
||||
isnothing(m) && return "", text
|
||||
return m[:header], replace(text, HEADER_REGEX => ""; count = 1)
|
||||
end
|
||||
|
||||
# HACK:
|
||||
# YAML.jl can't parse text including ``` characters, so first replace all the inline code
|
||||
# with these temporary code start/end string
|
||||
const HEADER_INLINE_START = "<weave_header_inline_start>"
|
||||
const HEADER_INLINE_END = "<weave_header_inline_end>"
|
||||
|
||||
function parse_header(header_text)
|
||||
isempty(header_text) && return Dict()
|
||||
pat = INLINE_REGEX => SubstitutionString("$(HEADER_INLINE_START)\\1$(HEADER_INLINE_END)")
|
||||
header_text = replace(header_text, pat)
|
||||
return YAML.load(header_text)
|
||||
end
|
||||
|
||||
include("markdown.jl")
|
||||
include("script.jl")
|
||||
|
|
44
src/run.jl
44
src/run.jl
|
@ -100,6 +100,8 @@ function run_doc(
|
|||
executed = [executed; result_chunks]
|
||||
end
|
||||
|
||||
replace_header_inline!(doc, report, mod) # evaluate and replace inline code in header
|
||||
|
||||
doc.header_script = report.header_script
|
||||
doc.chunks = executed
|
||||
|
||||
|
@ -172,9 +174,7 @@ function run_chunk(chunk::DocChunk, doc::WeaveDoc, report::Report, SandBox::Modu
|
|||
return chunk
|
||||
end
|
||||
|
||||
function run_inline(inline::InlineText, doc::WeaveDoc, report::Report, SandBox::Module)
|
||||
return inline
|
||||
end
|
||||
run_inline(inline::InlineText, doc::WeaveDoc, report::Report, SandBox::Module) = inline
|
||||
|
||||
function run_inline(inline::InlineCode, doc::WeaveDoc, report::Report, SandBox::Module)
|
||||
# Make a temporary CodeChunk for running code. Collect results and don't wrap
|
||||
|
@ -237,17 +237,11 @@ function capture_output(expr, SandBox::Module, term, disp, lastline, throw_error
|
|||
reader = @async read(rw, String)
|
||||
try
|
||||
obj = Core.eval(SandBox, expr)
|
||||
if (term || disp) && (typeof(expr) != Expr || expr.head != :toplevel)
|
||||
isnothing(obj) || display(obj)
|
||||
# This shows images and lone variables, result can
|
||||
# Handle last line sepately
|
||||
elseif lastline && !isnothing(obj)
|
||||
(typeof(expr) != Expr || expr.head != :toplevel) && display(obj)
|
||||
end
|
||||
catch E
|
||||
throw_errors && throw(E)
|
||||
display(E)
|
||||
@warn("ERROR: $(typeof(E)) occurred, including output in Weaved document")
|
||||
!isnothing(obj) && ((term || disp) || lastline) && display(obj)
|
||||
catch err
|
||||
throw_errors && throw(err)
|
||||
display(err)
|
||||
@warn "ERROR: $(typeof(err)) occurred, including output in Weaved document"
|
||||
finally
|
||||
redirect_stdout(oldSTDOUT)
|
||||
close(wr)
|
||||
|
@ -486,3 +480,25 @@ function collect_results(chunk::CodeChunk, fmt::CollectResult)
|
|||
end
|
||||
return [chunk]
|
||||
end
|
||||
|
||||
const HEADER_INLINE = Regex("$(HEADER_INLINE_START)(?<code>.+)$(HEADER_INLINE_END)")
|
||||
|
||||
replace_header_inline!(doc, report, mod) = _replace_header_inline!(doc, doc.header, report, mod)
|
||||
|
||||
function _replace_header_inline!(doc, header, report, mod)
|
||||
replace!(header) do (k,v)
|
||||
return k => v isa Dict ?
|
||||
_replace_header_inline!(doc, v, report, mod) :
|
||||
replace(v, HEADER_INLINE => s -> begin
|
||||
m = match(HEADER_INLINE, s)
|
||||
run_inline_code(m[:code], doc, report, mod)
|
||||
end)
|
||||
end
|
||||
return header
|
||||
end
|
||||
|
||||
function run_inline_code(s, doc, report, mod)
|
||||
inline = InlineCode(s, 1, 1, 1, :inline)
|
||||
inline = run_inline(inline, doc, report, mod)
|
||||
return strip(inline.output, '"')
|
||||
end
|
||||
|
|
|
@ -9,7 +9,7 @@ mutable struct WeaveDoc
|
|||
chunks::Vector{WeaveChunk}
|
||||
cwd::AbstractString
|
||||
format::Any
|
||||
doctype::AbstractString
|
||||
doctype::String
|
||||
header_script::String
|
||||
header::Dict
|
||||
template::Union{AbstractString,Mustache.MustacheTokens}
|
||||
|
|
|
@ -1,8 +1,3 @@
|
|||
---
|
||||
options :
|
||||
out_path : gadfly
|
||||
---
|
||||
|
||||
~~~~{.julia}
|
||||
using Gadfly
|
||||
x = range(0, stop =2π, step=0.05)
|
||||
|
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
````julia
|
||||
using Gadfly
|
||||
x = range(0, stop =2π, step=0.05)
|
||||
|
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
````julia
|
||||
using Gadfly
|
||||
x = range(0, stop =2π, step=0.05)
|
||||
|
|
|
@ -1,8 +1,3 @@
|
|||
---
|
||||
options :
|
||||
out_path : gadfly
|
||||
---
|
||||
|
||||
~~~~{.julia}
|
||||
using Gadfly
|
||||
x = range(0, stop =2π, step=0.05)
|
||||
|
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
````julia
|
||||
using Gadfly
|
||||
x = range(0, stop =2π, step=0.05)
|
||||
|
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
\begin{juliacode}
|
||||
using Gadfly
|
||||
x = range(0, stop =2π, step=0.05)
|
||||
|
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
\begin{juliacode}
|
||||
using Gadfly
|
||||
x = range(0, stop =2π, step=0.05)
|
||||
|
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
\begin{juliacode}
|
||||
using Gadfly
|
||||
x = range(0, stop =2π, step=0.05)
|
||||
|
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
\begin{juliacode}
|
||||
using Gadfly
|
||||
x = range(0, stop =2π, step=0.05)
|
||||
|
|
|
@ -1,7 +1,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
<div class="Random plot">
|
||||
|
||||
<p>Some inline output</p>
|
||||
|
|
|
@ -1,7 +1,6 @@
|
|||
|
||||
|
||||
|
||||
|
||||
\begin{frame}[fragile]
|
||||
\frametitle{Random plot}
|
||||
|
||||
|
|
|
@ -51,33 +51,6 @@ doc.template = "templates/mini.tpl"
|
|||
rendered = Weave.render_doc("Hello", doc)
|
||||
@test rendered == "\nHello\n"
|
||||
|
||||
# Test header parsing and stripping
|
||||
header = """
|
||||
---
|
||||
title : Test block
|
||||
author : Matti Pastell
|
||||
---
|
||||
|
||||
# Actual header
|
||||
|
||||
and some text
|
||||
|
||||
"""
|
||||
|
||||
dchunk = Weave.DocChunk(header, 1, 1)
|
||||
h = Weave.parse_header(dchunk)
|
||||
h_ref = Dict("author" => "Matti Pastell", "title" => "Test block")
|
||||
@test h_ref == h
|
||||
|
||||
Weave.strip_header!(dchunk, "md2html")
|
||||
h_ref = """
|
||||
# Actual header
|
||||
|
||||
and some text
|
||||
|
||||
"""
|
||||
@test dchunk.content[1].content == h_ref
|
||||
|
||||
# Test wrapping
|
||||
|
||||
cows = repeat("🐄", 100)
|
||||
|
|
|
@ -8,7 +8,7 @@ function test_gadfly(doctype, fig_ext)
|
|||
out_path = joinpath(@__DIR__ , "documents/gadfly/"),
|
||||
doctype = doctype, fig_ext = fig_ext)
|
||||
result = read(out, String)
|
||||
#cp(out, out*fig_ext*"."*doctype, force=true) # Used when adding new tests
|
||||
# cp(out, out*fig_ext*"."*doctype, force=true) # Used when adding new tests
|
||||
ref = read(out*fig_ext*"."*doctype, String)
|
||||
@test result == ref
|
||||
rm(out)
|
||||
|
|
|
@ -12,7 +12,7 @@ Some markdown with inline stuff and `j code`
|
|||
|
||||
"""
|
||||
|
||||
ms = collect(eachmatch(Weave.INLINE_REGEX, doc))
|
||||
ms = collect(eachmatch(Weave.INLINE_REGEXES, doc))
|
||||
@test ms[1][2] == "println(\"Something\")"
|
||||
@test ms[2][1] == "code"
|
||||
@test ms[3][1] == "show(\"is\")"
|
||||
|
|
|
@ -1,12 +1,16 @@
|
|||
using Weave, Test
|
||||
using Weave: run_doc
|
||||
using Weave: WeaveDoc, run_doc
|
||||
|
||||
|
||||
# TODO: add test for header processsing
|
||||
# TODO: add test for `include_weave`
|
||||
|
||||
# constructs `WeaveDoc` from `String`
|
||||
mock_doc(str, chunk_parser = Weave.parse_markdown) = Weave.WeaveDoc("dummy", chunk_parser(str))
|
||||
function mock_doc(str, format = "markdown")
|
||||
f = tempname()
|
||||
write(f, str)
|
||||
return WeaveDoc(f, format)
|
||||
end
|
||||
|
||||
|
||||
@testset "Weave" begin
|
||||
|
|
|
@ -1,26 +1,114 @@
|
|||
using Weave.YAML
|
||||
using Weave: separate_header_text, parse_header, specific_options!
|
||||
|
||||
# TODO:
|
||||
# - header stripping
|
||||
# - header parsing from a Weave document
|
||||
# TODO: add test for header restoring (strip)
|
||||
|
||||
@testset "header separation" begin
|
||||
|
||||
header_body = """
|
||||
options:
|
||||
foo: bar
|
||||
"""
|
||||
|
||||
let
|
||||
header_text = "---\n$header_body---"
|
||||
f, l = separate_header_text("$header_text")
|
||||
@test occursin(header_body, f)
|
||||
@test isempty(l)
|
||||
end
|
||||
|
||||
let
|
||||
doc_body = "hogehoge"
|
||||
header_text = "---\n$header_body---\n$doc_body"
|
||||
f, l = separate_header_text("$header_text")
|
||||
@test occursin(header_body, f)
|
||||
@test occursin(doc_body, l)
|
||||
end
|
||||
|
||||
let
|
||||
slide_body = """
|
||||
---
|
||||
slide comes here !
|
||||
---
|
||||
"""
|
||||
header_text = "---\n$header_body---\n$slide_body"
|
||||
f, l = separate_header_text("$header_text")
|
||||
@test occursin(header_body, f)
|
||||
@test occursin(slide_body, l)
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
|
||||
header = YAML.load("""
|
||||
@testset "dynamic header specifications" begin
|
||||
|
||||
let
|
||||
d = mock_doc("""
|
||||
---
|
||||
title: No. `j 1`
|
||||
---
|
||||
""")
|
||||
run_doc(d)
|
||||
@test d.header["title"] == "No. 1"
|
||||
end
|
||||
|
||||
let
|
||||
m = Core.eval(Main, :(module $(gensym(:WeaveTest)) end))
|
||||
|
||||
# run in target module
|
||||
@eval m n = 1
|
||||
d = mock_doc("""
|
||||
---
|
||||
title: No. `j n`
|
||||
---
|
||||
""")
|
||||
run_doc(d; mod = m)
|
||||
@test d.header["title"] == "No. 1"
|
||||
|
||||
# strip quotes by default
|
||||
@eval m s = "1"
|
||||
d = mock_doc("""
|
||||
---
|
||||
title: No. `j s`
|
||||
---
|
||||
""")
|
||||
run_doc(d; mod = m)
|
||||
@test d.header["title"] == "No. 1" # otherwise `"No. "1""`
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
|
||||
@testset "doctype specific header configuration" begin
|
||||
|
||||
header = parse_header("""
|
||||
---
|
||||
options:
|
||||
out_path: reports
|
||||
out_path: reports # should be overwrote
|
||||
md2html:
|
||||
out_path : html/
|
||||
md2pdf:
|
||||
out_path : pdf/
|
||||
github:
|
||||
out_path : md/
|
||||
fig_ext : .png
|
||||
fig_ext : .png # should remain
|
||||
---
|
||||
""")
|
||||
|
||||
let args = header[Weave.WEAVE_OPTION_NAME]
|
||||
@test Weave.combine_args(args, "md2html") == Dict("fig_ext" => ".png", "out_path" => "html/")
|
||||
@test Weave.combine_args(args, "github") == Dict("fig_ext" => ".png", "out_path" => "md/")
|
||||
@test Weave.combine_args(args, "pandoc") == Dict("fig_ext" => ".png", "out_path" => "reports")
|
||||
weave_options = header[Weave.WEAVE_OPTION_NAME]
|
||||
|
||||
let md2html_options = copy(weave_options)
|
||||
specific_options!(md2html_options, "md2html")
|
||||
@test md2html_options == Dict("fig_ext" => ".png", "out_path" => "html/")
|
||||
end
|
||||
|
||||
let md2pdf_options = copy(weave_options)
|
||||
specific_options!(md2pdf_options, "md2pdf")
|
||||
@test md2pdf_options == Dict("fig_ext" => ".png", "out_path" => "pdf/")
|
||||
end
|
||||
|
||||
let github_options = copy(weave_options)
|
||||
specific_options!(github_options, "github")
|
||||
@test github_options == Dict("fig_ext" => ".png", "out_path" => "md/")
|
||||
end
|
||||
|
||||
end
|
||||
|
|
Loading…
Reference in New Issue