Merge pull request #329 from JunoLab/avi/dynamic

dynamic YAML parsing
pull/331/head
Shuhei Kadowaki 2020-05-16 18:57:07 +09:00 committed by GitHub
commit 11c720ad7f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
30 changed files with 390 additions and 290 deletions

View File

@ -12,6 +12,7 @@ makedocs(
"getting_started.md",
"usage.md",
"publish.md",
"header.md",
"chunk_options.md",
"notebooks.md",
"function_index.md",

View File

@ -8,6 +8,7 @@ Options are separated using ";" and need to be valid Julia expressions. Example:
Weave currently supports the following chunk options with the following defaults:
## Options for code
- `echo = true`: Echo the code in the output document. If `false` the source code will be hidden.
@ -21,6 +22,7 @@ Weave currently supports the following chunk options with the following defaults
- `hold = false`: Hold all results until the end of the chunk.
- `tangle = true`: Set tangle to `false` to exclude chunk from tangled code.
## Options for figures
- `fig_width = 6`: Figure width passed to plotting library.
@ -42,7 +44,7 @@ You can set the default chunk options (and `weave` arguments) for a document usi
```yaml
---
options:
out_width : 50%
out_width : 50%
---
```

88
doc/src/header.md Normal file
View File

@ -0,0 +1,88 @@
# Header Configuration
When `weave`ing a markdown document, you use YAML header to provide additional metadata and configuration options.
A YAML header should be in the beginning of the input document delimited with `---`.
!!! warning
YAML header configuration is only supported when `weave`ing [Julia markdown documents](@ref document-syntax).
## Document Metadata
You can set additional document metadata in YAML header.
When `weave`ing to Julia markdown documents to HTML or PDF, Weave respects the following metadata specification:
- `title`
- `author`
- `date`
An example:
```yaml
---
title : Header Example
author : Shuhei Kadowaki
date: 16th May 2020
---
```
!!! note
You can also have other metadata, but they won't appear in the resulting HTML and PDF.
If you weave to Julia markdown to GitHub/Hugo markdown, all the metadata will be preserved.
### Dynamic Metadata
The metadata can be "dynamic"; if you have [inline code](@ref) within YAML header, they will be evaluated _after_ evaluating all the chunks and replaced with the results.
The example document below will set `date` metadata dynamically.
Note that `Date` is available since the chunk is evaluated first.
```md
---
title : Header Example
author : Shuhei Kadowaki
date: `j Date(now())`
---
```julia; echo = false
using Datas
```
```
## Configuration Options
Each of keyword arguments of [`weave`](@ref) can be set in the YAML header under `options` field.
You can also set [Chunks Options](@ref) there that will be applied globally.
The example below sets `out_path` and `doctype` options and overwrites `term` and `wrap` chunk options:
```yaml
---
title : Header Example
author : Shuhei Kadowaki
date: 16th May 2020
options:
out_path: relative/path/to/this/document
doctype: github
term: true
wrap: false
---
```
!!! note
- configurations specified within the YAML header have higher precedence than those specified via `weave` keyword arguments
- chunk options specified within each chunk have higher precedence than the global global chunk options specified within the YAML header
## Format Specific Options
The header configurations can be format specific.
Here is how to set different `out_path` for `md2html` and `md2pdf` and set `fig_ext` globally:
```yaml
---
options:
md2html:
out_path : html
md2pdf:
out_path : pdf
fig_ext : .png
---
```

View File

@ -1,4 +1,3 @@
# Weave.jl - Scientific Reports Using Julia
This is the documentation of [Weave.jl](http://github.com/mpastell/weave.jl).
@ -22,10 +21,18 @@ and [Sweave](https://stat.ethz.ch/R-manual/R-patched/library/utils/doc/Sweave.pd
![Weave in Juno demo](https://user-images.githubusercontent.com/40514306/76081328-32f41900-5fec-11ea-958a-375f77f642a2.png)
## Contents
## Index
```@contents
Pages = ["getting_started.md", "usage.md",
"publish.md", "chunk_options.md", "notebooks.md",
"function_index.md"]
Pages = [
"index.md",
"getting_started.md",
"usage.md",
"publish.md",
"header.md",
"chunk_options.md",
"notebooks.md",
"function_index.md",
]
```

View File

@ -3,7 +3,7 @@
You can write your documentation and code in input document using Markdown, Noweb or script
syntax and use [`weave`](@ref) function to execute to document to capture results and figures.
## Weave
## `weave`
Weave document with markup and julia code using `Plots.jl` for plots,
`out_path = :pwd` makes the results appear in the current working directory.
@ -21,7 +21,7 @@ weave(joinpath(dirname(pathof(Weave)), "../examples", "FIR_design.jmd"), out_pat
weave
```
## Tangle
## `tangle`
Tangling extracts the code from document:
@ -29,7 +29,7 @@ Tangling extracts the code from document:
tangle
```
## Supported output formats
## Supported Output Formats
Weave automatically detects the output format based on the file extension.
The auto output format detection is handled by `detect_doctype(path::AbstractString)`:
@ -59,7 +59,7 @@ using Weave # hide
list_out_formats()
```
## Document syntax
## [Document Syntax](@id document-syntax)
Weave uses markdown, Noweb or script syntax for defining the code chunks and
documentation chunks. You can also weave Jupyter notebooks. The format is detected based on the file extension, but you can also set it manually using the `informat` parameter.
@ -73,13 +73,13 @@ ext == ".ipynb" && return "notebook"
return "noweb"
```
## Documentation chunks
## Documentation Chunks
In Markdown and Noweb input formats documentation chunks are the parts that aren't inside code delimiters. Documentation chunks can be written with several different markup languages.
## Code chunks
## Code Chunks
### Markdown format
### Markdown Format
Markdown code chunks are defined using fenced code blocks with options following on the same line. e.g. to hide code from output you can use:
@ -89,7 +89,7 @@ Markdown code chunks are defined using fenced code blocks with options following
[Sample document]( https://github.com/mpastell/Weave.jl/blob/master/examples/FIR_design.jmd)
## Inline code
## [Inline Code](@id inline-code)
You can also add inline code to your documents using
@ -121,60 +121,38 @@ or to produce any html output:
```
### Noweb format
### Noweb Format
Code chunks start with a line marked with `<<>>=` or `<<options>>=` and end with line marked with `@`. The code between the start and end markers is executed and the output is captured to the output document. See [chunk options](../chunk_options/).
### Script format
### Script Format
Weave also support script input format with a markup in comments.
These scripts can be executed normally using Julia or published with
Weave. Documentation is in lines starting with
`#'`, `#%%` or `# %%`, and code is executed and results are included
in the weaved document.
These scripts can be executed normally using Julia or published with Weave.
All lines that are not documentation are treated as code. You can set chunk options
using lines starting with `#+` just before code e.g. `#+ term=true`.
Lines starting with `#'`, `#%%` or `# %%` are treated as document.
The format is identical to [Pweave](http://mpastell.com/pweave/pypublish.html)
and the concept is similar to publishing documents with MATLAB or
using Knitr's [spin](http://yihui.name/knitr/demo/stitch/).
All non-document lines are treated as code.
You can set chunk options using lines starting with `#+` just before code e.g:
```julia
#+ term=true
hoge # some code comes here
```
The format is identical to [Pweave](http://mpastell.com/pweave/pypublish.html) and the concept is similar to publishing documents with MATLAB or using Knitr's [spin](http://yihui.name/knitr/demo/stitch/).
Weave will remove the first empty space from each line of documentation.
[See sample document:](https://github.com/mpastell/Weave.jl/blob/master/examples/FIR_design.jl)
## Setting document options in header
You can use a YAML header in the beginning of the input document delimited with "---" to set the document title, author and date e.g. and default document options. Each of Weave command line arguments and chunk options can be set in header using `options` field. Below is an example that sets document `out_path` and `doctype` using the header.
## Configuration via YAML Header
When `weave`ing markdown files, you use YAML header to provide additional metadata and configuration options.
See [Header Configuration](@ref) section for more details.
```yaml
---
title : Weave example
author : Matti Pastell
date: 15th December 2016
options:
out_path : reports/example.md
doctype : github
---
```
You can also set format specific options. Here is how to set different `out_path` for `md2html` and `md2pdf` and set `fig_ext` for both:
```
---
options:
md2html:
out_path : html
md2pdf:
out_path : pdf
fig_ext : .png
---
```
## Passing arguments to documents
## Passing Runtime Arguments to Documents
You can pass arguments as `Dict` to the weaved document using the `args` argument
to `weave`. The arguments will be available as `WEAVE_ARGS` variable in the document.
@ -192,18 +170,18 @@ weave("mydoc.jmd", args = Dict("filename" => "somedata.h5"))
and you can access the filename from document as follows:
```
```julia
print(WEAVE_ARGS["filename"])
```
```julia
print(WEAVE_ARGS["filename"])
```
```
You can use the `out_path` argument to control the name of the
output document.
## Include Weave document in Julia
You can call `include_weave` on a Weave document to run the contents
of all code chunks in Julia.
## `include_weave`
You can call `include_weave` on a Weave document and run all code chunks within in the current session.
```@docs
include_weave

View File

@ -3,6 +3,8 @@ module Weave
using Highlights, Mustache, Requires
const PKG_DIR = normpath(@__DIR__, "..")
const TEMPLATE_DIR = normpath(PKG_DIR, "templates")
const WEAVE_OPTION_NAME = "options" # TODO: rename to "weave_options"
function __init__()
@ -114,8 +116,7 @@ function weave(
latex_cmd::AbstractString = "xelatex",
latex_keep_unicode::Bool = false,
)
doc = WeaveDoc(source, informat)
isnothing(doctype) && (doctype = detect_doctype(doc.source))
doc = WeaveDoc(source, informat, doctype)
# overwrites given options with header options, which have more precedence
# NOTE:
@ -124,7 +125,7 @@ function weave(
weave_options = get(doc.header, WEAVE_OPTION_NAME, Dict())
if !isempty(weave_options)
doctype = get(weave_options, "doctype", doctype)
weave_options = combine_args(weave_options, doctype)
specific_options!(weave_options, doctype)
if haskey(weave_options, "out_path")
out_path = let
out_path = weave_options["out_path"]
@ -201,6 +202,16 @@ function weave(
return abspath(outname)
end
function specific_options!(weave_options, doctype)
fmts = keys(formats)
for (k,v) in weave_options
if k in fmts
k == doctype && merge!(weave_options, v)
delete!(weave_options, k)
end
end
end
weave(doc::AbstractString, doctype::Union{Symbol,AbstractString}) =
weave(doc; doctype = doctype)

View File

@ -62,18 +62,3 @@ get_chunk_defaults() = rcParams[:chunk_defaults]
Restore Weave.jl default chunk options.
"""
restore_chunk_defaults!() = rcParams[:chunk_defaults] = defaultParams[:chunk_defaults]
"""Combine format specific and common options from document header"""
function combine_args(args, doctype)
common = Dict()
specific = Dict()
for key in keys(args)
if key in keys(Weave.formats)
specific[key] = args[key]
else
common[key] = args[key]
end
end
haskey(specific, doctype) && merge!(common, specific[doctype])
common
end

View File

@ -92,19 +92,6 @@ function convert_to_notebook(doc)
cells = []
ex_count = 1
# Handle header
head_tpl = """
{{#:title}}# {{:title}}{{/:title}}
{{#:author}}### {{{:author}}}{{/:author}}
{{#:date}}### {{{:date}}}{{/:date}}
"""
if isa(doc.chunks[1], DocChunk)
strip_header!(doc)
doc.chunks[1].content[1].content =
Mustache.render(head_tpl; [Pair(Symbol(k), v) for (k, v) in doc.header]...) * doc.chunks[1].content[1].content
end
for chunk in doc.chunks
if isa(chunk, DocChunk)
push!(

View File

@ -1,5 +1,4 @@
using Mustache, Highlights
using .WeaveMarkdown, Markdown, Dates
using Mustache, Highlights, .WeaveMarkdown, Markdown, Dates
using REPL.REPLCompletions: latex_symbols
function format(doc::WeaveDoc)
@ -18,7 +17,7 @@ function format(doc::WeaveDoc)
docformat.formatdict[:cwd] = doc.cwd # pass wd to figure formatters
docformat.formatdict[:theme] = doc.highlight_theme
strip_header!(doc)
restore_header!(doc)
for chunk in copy(doc.chunks)
result = format_chunk(chunk, formatdict, docformat)
@ -36,11 +35,11 @@ render_doc(formatted, doc, format) = formatted
function highlight(
mime::MIME,
source::AbstractString,
output,
lexer,
theme = Highlights.Themes.DefaultTheme,
)
return sprint((io, x) -> Highlights.highlight(io, mime, x, lexer, theme), source)
return sprint((io, x) -> Highlights.highlight(io, mime, x, lexer, theme), output)
end
function stylesheet(m::MIME, theme)
@ -54,22 +53,14 @@ function render_doc(formatted, doc, format::JMarkdown2HTML)
wversion = ""
wtime = string(Date(now()))
if isempty(doc.css)
theme_css =
read(joinpath(dirname(@__FILE__), "../templates/skeleton_css.css"), String)
else
theme_css = read(doc.css, String)
end
theme_path = isempty(doc.css) ? normpath(TEMPLATE_DIR, "skeleton_css.css") : doc.css
theme_css = read(theme_path, String)
if isa(doc.template, Mustache.MustacheTokens)
template = doc.template
elseif isempty(doc.template)
template = Mustache.template_from_file(joinpath(
dirname(@__FILE__),
"../templates/julia_html.tpl",
))
template = if isa(doc.template, Mustache.MustacheTokens)
doc.template
else
template = Mustache.template_from_file(doc.template)
template_path = isempty(doc.template) ? normpath(TEMPLATE_DIR, "julia_html.tpl") : doc.template
Mustache.template_from_file(template_path)
end
return Mustache.render(
@ -92,15 +83,11 @@ function render_doc(formatted, doc, format::JMarkdown2tex)
wversion = ""
wtime = string(Date(now()))
if isa(doc.template, Mustache.MustacheTokens)
template = doc.template
elseif isempty(doc.template)
template = Mustache.template_from_file(joinpath(
dirname(@__FILE__),
"../templates/julia_tex.tpl",
))
template = if isa(doc.template, Mustache.MustacheTokens)
doc.template
else
template = Mustache.template_from_file(doc.template)
template_path = isempty(doc.template) ? normpath(TEMPLATE_DIR, "julia_tex.tpl") : doc.template
Mustache.template_from_file(template_path)
end
return Mustache.render(
@ -111,40 +98,27 @@ function render_doc(formatted, doc, format::JMarkdown2tex)
)
end
strip_header!(doc::WeaveDoc) = strip_header!(doc.chunks[1], doc.doctype)
function strip_header!(docchunk::DocChunk, doctype)
doctype == "pandoc" && return
content = docchunk.content[1].content
if (m = match(HEADER_REGEX, content)) !== nothing
# TODO: is there other format where we want to keep headers ?
docchunk.content[1].content = if doctype != "github"
lstrip(replace(content, HEADER_REGEX => ""))
else
# only strips Weave headers
header = YAML.load(m[:header])
delete!(header, WEAVE_OPTION_NAME)
if isempty(header)
lstrip(replace(content, HEADER_REGEX => ""))
else
lstrip(replace(content, HEADER_REGEX => "---\n$(YAML.write(header))---"))
end
end
end
end
strip_header!(codechunk::CodeChunk, doctype) = return
function restore_header!(doc)
# TODO: is there any other format where we want to restore headers ?
doc.doctype "github" && return
function format_chunk(chunk::DocChunk, formatdict, docformat)
return join([format_inline(c) for c in chunk.content], "")
# only strips Weave headers
delete!(doc.header, WEAVE_OPTION_NAME)
isempty(doc.header) && return
# restore remained headers as `DocChunk`
header_text = "---\n$(YAML.write(doc.header))---"
pushfirst!(doc.chunks, DocChunk(header_text, 0, 0))
end
function format_inline(inline::InlineText)
return inline.content
end
format_chunk(chunk::DocChunk, formatdict, docformat) = join((format_inline(c) for c in chunk.content))
format_inline(inline::InlineText) = inline.content
function format_inline(inline::InlineCode)
isempty(inline.rich_output) || return inline.rich_output
isempty(inline.figures) || return inline.figures[end]
isempty(inline.output) || return inline.output
return inline.output
end
function ioformat!(io::IOBuffer, out::IOBuffer, fun = WeaveMarkdown.latex)
@ -155,10 +129,7 @@ function ioformat!(io::IOBuffer, out::IOBuffer, fun = WeaveMarkdown.latex)
end
end
function addspace(op, inline)
inline.ctype == :line && (op = "\n$op\n")
return op
end
addspace(op, inline) = (inline.ctype === :line && (op = "\n$op\n"); op)
function format_chunk(chunk::DocChunk, formatdict, docformat::JMarkdown2tex)
out = IOBuffer()
@ -275,15 +246,11 @@ function format_chunk(chunk::CodeChunk, formatdict, docformat)
return result
end
function format_output(result::AbstractString, docformat)
return result
end
format_output(result, docformat) = result
function format_output(result::AbstractString, docformat::JMarkdown2HTML)
return Markdown.htmlesc(result)
end
format_output(result, docformat::JMarkdown2HTML) = Markdown.htmlesc(result)
function format_output(result::AbstractString, docformat::JMarkdown2tex)
function format_output(result, docformat::JMarkdown2tex)
# Highligts has some extra escaping defined, eg of $, ", ...
result_escaped = sprint(
(io, x) ->
@ -294,11 +261,9 @@ function format_output(result::AbstractString, docformat::JMarkdown2tex)
return result_escaped
end
function format_code(result::AbstractString, docformat)
return result
end
format_code(result, docformat) = result
function format_code(result::AbstractString, docformat::JMarkdown2tex)
function format_code(result, docformat::JMarkdown2tex)
highlighted = highlight(
MIME("text/latex"),
strip(result),
@ -339,7 +304,7 @@ function texify(s)
return ts
end
function format_code(result::AbstractString, docformat::JMarkdown2HTML)
function format_code(result, docformat::JMarkdown2HTML)
return highlight(
MIME("text/html"),
strip(result),
@ -348,7 +313,7 @@ function format_code(result::AbstractString, docformat::JMarkdown2HTML)
)
end
function format_code(result::AbstractString, docformat::Pandoc2HTML)
function format_code(result, docformat::Pandoc2HTML)
return highlight(
MIME("text/html"),
strip(result),
@ -409,9 +374,7 @@ function format_termchunk(chunk, formatdict, docformat::JMarkdown2tex)
return result
end
function indent(text, nindent)
return join(map(x -> string(repeat(" ", nindent), x), split(text, "\n")), "\n")
end
indent(text, nindent) = join(map(x -> string(repeat(' ', nindent), x), split(text, '\n')), '\n')
function wraplines(text, line_width = 75)
result = AbstractString[]

View File

@ -509,7 +509,7 @@ function formatfigures(chunk, docformat::AsciiDoc)
end
# Add new supported formats here
const formats = Dict{AbstractString,Any}(
const formats = Dict(
"tex" => tex,
"texminted" => texminted,
"pandoc" => pandoc,

View File

@ -1,6 +1,5 @@
"""
`pandoc2html(formatted::AbstractString, doc::WeaveDoc)`
pandoc2html(formatted::AbstractString, doc::WeaveDoc)
Convert output from pandoc markdown to html using Weave.jl template
"""
@ -38,10 +37,6 @@ function pandoc2html(
html = ""
outname = basename(outname)
open("temp.md", "w") do io
println(io, formatted)
end
try
cmd = `pandoc -f markdown+raw_html -s --mathjax=""
$filt $citeproc $pandoc_options
@ -63,7 +58,7 @@ function pandoc2html(
end
"""
`pandoc2pdf(formatted::AbstractString, doc::WeaveDoc)`
pandoc2pdf(formatted::AbstractString, doc::WeaveDoc)
Convert output from pandoc markdown to pdf using Weave.jl template
"""
@ -95,7 +90,7 @@ function pandoc2pdf(
citeproc = []
end
@info("Done executing code. Running xelatex")
@info "Done executing code. Running xelatex"
try
cmd = `pandoc -f markdown+raw_tex -s --pdf-engine=xelatex --highlight-style=tango
$filt $citeproc $pandoc_options
@ -117,7 +112,7 @@ function run_latex(doc::WeaveDoc, outname, latex_cmd = "xelatex")
old_wd = pwd()
cd(doc.cwd)
xname = basename(outname)
@info("Weaved code to $outname . Running $latex_cmd") # space before '.' added for link to be clickable in Juno terminal
@info "Weaved code to $outname . Running $latex_cmd" # space before '.' added for link to be clickable in Juno terminal
textmp = mktempdir(".")
try
out = read(

View File

@ -1,19 +1,16 @@
using JSON, YAML
function WeaveDoc(source, format::Union{Nothing,AbstractString} = nothing)
document = replace(read(source, String), "\r\n" => "\n") # normalize line ending
isnothing(format) && (format = detect_informat(source))
chunks = parse_doc(document, format)
return WeaveDoc(source, chunks)
end
function WeaveDoc(source, chunks::Vector{WeaveChunk})
function WeaveDoc(source, informat = nothing, doctype = nothing)
path, fname = splitdir(abspath(source))
basename = splitext(fname)[1]
header = parse_header(first(chunks))
# get chunk defaults from header and update
isnothing(informat) && (informat = detect_informat(source))
header, chunks = parse_doc(read(source, String), informat)
isnothing(doctype) && (doctype = detect_doctype(source))
# update default chunk options from header
chunk_defaults = deepcopy(get_chunk_defaults())
if haskey(header, WEAVE_OPTION_NAME)
for key in keys(chunk_defaults)
@ -30,7 +27,7 @@ function WeaveDoc(source, chunks::Vector{WeaveChunk})
chunks,
"",
nothing,
"",
doctype,
"",
header,
"",
@ -42,12 +39,12 @@ function WeaveDoc(source, chunks::Vector{WeaveChunk})
end
"""
detect_informat(source::AbstractString)
detect_informat(path)
Detect Weave input format based on file extension of `source`.
Detect Weave input format based on file extension of `path`.
"""
function detect_informat(source::AbstractString)
ext = lowercase(last(splitext(source)))
function detect_informat(path)
ext = lowercase(last(splitext(path)))
ext == ".jl" && return "script"
ext == ".jmd" && return "markdown"
@ -55,18 +52,17 @@ function detect_informat(source::AbstractString)
return "noweb"
end
function parse_doc(document, format)::Vector{WeaveChunk}
return if format == "markdown"
parse_markdown(document)
elseif format == "noweb"
parse_markdown(document, true)
elseif format == "script"
parse_script(document)
elseif format == "notebook"
parse_notebook(document)
else
error("unsupported format given: $(format)")
end
function parse_doc(document, informat)
document = replace(document, "\r\n" => "\n") # normalize line ending
header_text, document = separate_header_text(document)
return parse_header(header_text),
informat == "markdown" ? parse_markdown(document) :
informat == "noweb" ? parse_markdown(document, true) :
informat == "script" ? parse_script(document) :
informat == "notebook" ? parse_notebook(document) :
error("unsupported input format given: $informat")
end
function pushopt(options::Dict, expr::Expr)
@ -75,6 +71,22 @@ function pushopt(options::Dict, expr::Expr)
end
end
"""
detect_doctype(path)
Detect the output format based on file extension.
"""
function detect_doctype(path)
_, ext = lowercase.(splitext(path))
match(r"^\.(jl|.?md|ipynb)", ext) !== nothing && return "md2html"
ext == ".rst" && return "rst"
ext == ".tex" && return "texminted"
ext == ".txt" && return "asciidoc"
return "pandoc"
end
# inline
# ------
@ -84,12 +96,13 @@ function DocChunk(text::AbstractString, number, start_line; notebook = false)
return DocChunk(content, number, start_line)
end
const INLINE_REGEX = r"`j\s+(.*?)`|^!\s(.*)$"m
const INLINE_REGEX = r"`j\s+(.*?)`"
const INLINE_REGEXES = r"`j\s+(.*?)`|^!\s(.*)$"m
function parse_inlines(text)::Vector{Inline}
occursin(INLINE_REGEX, text) || return parse_inline(text)
occursin(INLINE_REGEXES, text) || return parse_inline(text)
inline_chunks = eachmatch(INLINE_REGEX, text)
inline_chunks = eachmatch(INLINE_REGEXES, text)
s = 1
e = 1
res = Inline[]
@ -118,20 +131,28 @@ parse_inline(text) = Inline[InlineText(text, 1, length(text), 1)]
# headers
# -------
parse_header(chunk::CodeChunk) = Dict()
const HEADER_REGEX = r"^---$(?<header>((?!---).)+)^---$"ms
function parse_header(chunk::DocChunk)
m = match(HEADER_REGEX, chunk.content[1].content)
if m !== nothing
header = YAML.load(string(m[:header]))
else
header = Dict()
end
return header
# TODO: non-Weave headers should keep live in a doc
# separates header section from `text`
function separate_header_text(text)
m = match(HEADER_REGEX, text)
isnothing(m) && return "", text
return m[:header], replace(text, HEADER_REGEX => ""; count = 1)
end
# HACK:
# YAML.jl can't parse text including ``` characters, so first replace all the inline code
# with these temporary code start/end string
const HEADER_INLINE_START = "<weave_header_inline_start>"
const HEADER_INLINE_END = "<weave_header_inline_end>"
function parse_header(header_text)
isempty(header_text) && return Dict()
pat = INLINE_REGEX => SubstitutionString("$(HEADER_INLINE_START)\\1$(HEADER_INLINE_END)")
header_text = replace(header_text, pat)
return YAML.load(header_text)
end
include("markdown.jl")
include("script.jl")

View File

@ -100,6 +100,8 @@ function run_doc(
executed = [executed; result_chunks]
end
replace_header_inline!(doc, report, mod) # evaluate and replace inline code in header
doc.header_script = report.header_script
doc.chunks = executed
@ -172,9 +174,7 @@ function run_chunk(chunk::DocChunk, doc::WeaveDoc, report::Report, SandBox::Modu
return chunk
end
function run_inline(inline::InlineText, doc::WeaveDoc, report::Report, SandBox::Module)
return inline
end
run_inline(inline::InlineText, doc::WeaveDoc, report::Report, SandBox::Module) = inline
function run_inline(inline::InlineCode, doc::WeaveDoc, report::Report, SandBox::Module)
# Make a temporary CodeChunk for running code. Collect results and don't wrap
@ -237,17 +237,11 @@ function capture_output(expr, SandBox::Module, term, disp, lastline, throw_error
reader = @async read(rw, String)
try
obj = Core.eval(SandBox, expr)
if (term || disp) && (typeof(expr) != Expr || expr.head != :toplevel)
isnothing(obj) || display(obj)
# This shows images and lone variables, result can
# Handle last line sepately
elseif lastline && !isnothing(obj)
(typeof(expr) != Expr || expr.head != :toplevel) && display(obj)
end
catch E
throw_errors && throw(E)
display(E)
@warn("ERROR: $(typeof(E)) occurred, including output in Weaved document")
!isnothing(obj) && ((term || disp) || lastline) && display(obj)
catch err
throw_errors && throw(err)
display(err)
@warn "ERROR: $(typeof(err)) occurred, including output in Weaved document"
finally
redirect_stdout(oldSTDOUT)
close(wr)
@ -486,3 +480,25 @@ function collect_results(chunk::CodeChunk, fmt::CollectResult)
end
return [chunk]
end
const HEADER_INLINE = Regex("$(HEADER_INLINE_START)(?<code>.+)$(HEADER_INLINE_END)")
replace_header_inline!(doc, report, mod) = _replace_header_inline!(doc, doc.header, report, mod)
function _replace_header_inline!(doc, header, report, mod)
replace!(header) do (k,v)
return k => v isa Dict ?
_replace_header_inline!(doc, v, report, mod) :
replace(v, HEADER_INLINE => s -> begin
m = match(HEADER_INLINE, s)
run_inline_code(m[:code], doc, report, mod)
end)
end
return header
end
function run_inline_code(s, doc, report, mod)
inline = InlineCode(s, 1, 1, 1, :inline)
inline = run_inline(inline, doc, report, mod)
return strip(inline.output, '"')
end

View File

@ -9,7 +9,7 @@ mutable struct WeaveDoc
chunks::Vector{WeaveChunk}
cwd::AbstractString
format::Any
doctype::AbstractString
doctype::String
header_script::String
header::Dict
template::Union{AbstractString,Mustache.MustacheTokens}

View File

@ -1,8 +1,3 @@
---
options :
out_path : gadfly
---
~~~~{.julia}
using Gadfly
x = range(0, stop =2π, step=0.05)

View File

@ -1,4 +1,3 @@
````julia
using Gadfly
x = range(0, stop =2π, step=0.05)

View File

@ -1,4 +1,3 @@
````julia
using Gadfly
x = range(0, stop =2π, step=0.05)

View File

@ -1,8 +1,3 @@
---
options :
out_path : gadfly
---
~~~~{.julia}
using Gadfly
x = range(0, stop =2π, step=0.05)

View File

@ -1,4 +1,3 @@
````julia
using Gadfly
x = range(0, stop =2π, step=0.05)

View File

@ -1,4 +1,3 @@
\begin{juliacode}
using Gadfly
x = range(0, stop =2π, step=0.05)

View File

@ -1,4 +1,3 @@
\begin{juliacode}
using Gadfly
x = range(0, stop =2π, step=0.05)

View File

@ -1,4 +1,3 @@
\begin{juliacode}
using Gadfly
x = range(0, stop =2π, step=0.05)

View File

@ -1,4 +1,3 @@
\begin{juliacode}
using Gadfly
x = range(0, stop =2π, step=0.05)

View File

@ -1,7 +1,6 @@
<div class="Random plot">
<p>Some inline output</p>

View File

@ -1,7 +1,6 @@
\begin{frame}[fragile]
\frametitle{Random plot}

View File

@ -51,33 +51,6 @@ doc.template = "templates/mini.tpl"
rendered = Weave.render_doc("Hello", doc)
@test rendered == "\nHello\n"
# Test header parsing and stripping
header = """
---
title : Test block
author : Matti Pastell
---
# Actual header
and some text
"""
dchunk = Weave.DocChunk(header, 1, 1)
h = Weave.parse_header(dchunk)
h_ref = Dict("author" => "Matti Pastell", "title" => "Test block")
@test h_ref == h
Weave.strip_header!(dchunk, "md2html")
h_ref = """
# Actual header
and some text
"""
@test dchunk.content[1].content == h_ref
# Test wrapping
cows = repeat("🐄", 100)

View File

@ -8,7 +8,7 @@ function test_gadfly(doctype, fig_ext)
out_path = joinpath(@__DIR__ , "documents/gadfly/"),
doctype = doctype, fig_ext = fig_ext)
result = read(out, String)
#cp(out, out*fig_ext*"."*doctype, force=true) # Used when adding new tests
# cp(out, out*fig_ext*"."*doctype, force=true) # Used when adding new tests
ref = read(out*fig_ext*"."*doctype, String)
@test result == ref
rm(out)

View File

@ -12,7 +12,7 @@ Some markdown with inline stuff and `j code`
"""
ms = collect(eachmatch(Weave.INLINE_REGEX, doc))
ms = collect(eachmatch(Weave.INLINE_REGEXES, doc))
@test ms[1][2] == "println(\"Something\")"
@test ms[2][1] == "code"
@test ms[3][1] == "show(\"is\")"

View File

@ -1,12 +1,16 @@
using Weave, Test
using Weave: run_doc
using Weave: WeaveDoc, run_doc
# TODO: add test for header processsing
# TODO: add test for `include_weave`
# constructs `WeaveDoc` from `String`
mock_doc(str, chunk_parser = Weave.parse_markdown) = Weave.WeaveDoc("dummy", chunk_parser(str))
function mock_doc(str, format = "markdown")
f = tempname()
write(f, str)
return WeaveDoc(f, format)
end
@testset "Weave" begin

View File

@ -1,26 +1,114 @@
using Weave.YAML
using Weave: separate_header_text, parse_header, specific_options!
# TODO:
# - header stripping
# - header parsing from a Weave document
# TODO: add test for header restoring (strip)
@testset "header separation" begin
header_body = """
options:
foo: bar
"""
let
header_text = "---\n$header_body---"
f, l = separate_header_text("$header_text")
@test occursin(header_body, f)
@test isempty(l)
end
let
doc_body = "hogehoge"
header_text = "---\n$header_body---\n$doc_body"
f, l = separate_header_text("$header_text")
@test occursin(header_body, f)
@test occursin(doc_body, l)
end
let
slide_body = """
---
slide comes here !
---
"""
header_text = "---\n$header_body---\n$slide_body"
f, l = separate_header_text("$header_text")
@test occursin(header_body, f)
@test occursin(slide_body, l)
end
end
header = YAML.load("""
@testset "dynamic header specifications" begin
let
d = mock_doc("""
---
title: No. `j 1`
---
""")
run_doc(d)
@test d.header["title"] == "No. 1"
end
let
m = Core.eval(Main, :(module $(gensym(:WeaveTest)) end))
# run in target module
@eval m n = 1
d = mock_doc("""
---
title: No. `j n`
---
""")
run_doc(d; mod = m)
@test d.header["title"] == "No. 1"
# strip quotes by default
@eval m s = "1"
d = mock_doc("""
---
title: No. `j s`
---
""")
run_doc(d; mod = m)
@test d.header["title"] == "No. 1" # otherwise `"No. "1""`
end
end
@testset "doctype specific header configuration" begin
header = parse_header("""
---
options:
out_path: reports
out_path: reports # should be overwrote
md2html:
out_path : html/
md2pdf:
out_path : pdf/
github:
out_path : md/
fig_ext : .png
fig_ext : .png # should remain
---
""")
let args = header[Weave.WEAVE_OPTION_NAME]
@test Weave.combine_args(args, "md2html") == Dict("fig_ext" => ".png", "out_path" => "html/")
@test Weave.combine_args(args, "github") == Dict("fig_ext" => ".png", "out_path" => "md/")
@test Weave.combine_args(args, "pandoc") == Dict("fig_ext" => ".png", "out_path" => "reports")
weave_options = header[Weave.WEAVE_OPTION_NAME]
let md2html_options = copy(weave_options)
specific_options!(md2html_options, "md2html")
@test md2html_options == Dict("fig_ext" => ".png", "out_path" => "html/")
end
let md2pdf_options = copy(weave_options)
specific_options!(md2pdf_options, "md2pdf")
@test md2pdf_options == Dict("fig_ext" => ".png", "out_path" => "pdf/")
end
let github_options = copy(weave_options)
specific_options!(github_options, "github")
@test github_options == Dict("fig_ext" => ".png", "out_path" => "md/")
end
end