| Age | Commit message (Collapse) | Author |
|---|
|
This is not clearly defined by the C99 standard.
Define ctype-like macros to force it to be ASCII / UTF-8 (not extended ASCII or
something like noticed on OpenBSD 3.8).
(In practise modern libc libraries are all ASCII and UTF-8-compatible. Otherwise
this would break many programs)
|
|
This also makes the programs exit with a non-zero status when a read or write
error occurs.
This makes checking the exit status more reliable in scripts.
A simple example to simulate a disk with no space left:
curl -s 'https://codemadness.org/atom.xml' | sfeed > f
/mnt/test: write failed, file system is full
echo $?
0
Which now produces:
curl -s 'https://codemadness.org/atom.xml' | sfeed > f
/mnt/test: write failed, file system is full
write error: <stdout>
echo $?
1
Tested with a small mfs on OpenBSD, fstab entry:
swap /mnt/test mfs rw,nodev,nosuid,-s=1M 0 0
|
|
This makes atleast feeds with simple ASCII work.
|
|
|
|
|
|
Also tested on MIPS32BE which has 32-bit time_t and which wraps the time value.
|
|
|
|
|
|
Most common-used compilers (gcc, clang) optimize this away though.
|
|
These are BSD functions.
- HaikuOS now compiles without having to use libbsd.
- Tested on SerenityOS (for fun), which doesn't have these functions (yet).
With a small change to support wcwidth() sfeed works on SerenityOS.
|
|
No functional difference because the base URI host is copied beforehand.
|
|
Removed/rewritten the functions:
absuri, parseuri, and encodeuri() for percent-encoding.
The functions are now split separately with the following purpose:
- uri_format: format struct uri into a string.
- uri_hasscheme: quick check if a string is absolute or not.
- uri_makeabs: make a URI absolute using a base uri and the original URI.
- uri_parse: parse a string into a struct uri.
The following URLs are better parsed:
- URLs with extra "/"'s in the path prepended are kept as is, no "/" is added
either for empty paths.
- URLs like "http://codemadness.org" are not changed to
"http://codemadness.org/" anymore (paths are kept as is, unless they are
non-empty and not start with "/").
- Paths are not percent-encoded anymore.
- URLs with userinfo field (username, password) are parsed.
like: ftp://user:password@[2001:db8::7]:2121/rfc/rfc1808.txt
- Non-authoritive URLs like mailto:some@email.org, magnet URIs, ISBN URIs/urn,
like: urn:isbn:0-395-36341-1 are allowed and parsed correctly.
- Both local (file:///) and non-local (file://) are supported.
- Specifying a base URL with a port will now only use it when the relative URL
has no host and port set and follows RFC3986 5.2.2 more closely.
- Parsing numeric port: parse as signed long and check <= 0, empty port is
allowed.
- Parsing URIs containing query, fragment, but no path separator (/) will now
parse the component properly.
For sfeed:
- Parse the baseURI only once (no need to do it every time for making absolute
URIs).
- If a link/enclosure is absolute already or if there is no base URL specified
then just print the link directly. There have also been other small performance
improvements related to handling URIs.
References:
- https://tools.ietf.org/html/rfc3986
- Section "5.2.2. Transform References" have also been helpful.
|
|
|
|
range >= 127
For example: "\xef\xbf\xb7" (codepoint 0xfff7), returns wcwidth(wc) == -1.
The next byte was incorrected seeked, but the codepoint itself was valid
(mbtowc).
|
|
|
|
characters
This affects sfeed_plain.
- Use unicode replacement character (codepoint 0xfffd) when a codepoint is
invalid and proceed printing the rest of the characters.
- When a codepoint is invalid reset the internal state of mbtowc(3), from the
OpenBSD man page:
" If a call to mbtowc() resulted in an undefined internal state, mbtowc()
must be called with s set to NULL to reset the internal state before it
can safely be used again."
- Optimize for the common ASCII case and use a macro to print the character
instead of a wasteful fwrite() function call. With 250k lines (+- 350MB) this
improves printing performance from 1.7s to 1.0s on my laptop. On an other
system it improved by +- 25%. Tested with clang and gcc and also tested the
worst-case (non-ASCII) with no penalty.
To test:
printf '0\tabc\xc3 def' | sfeed_plain
Before:
1970-01-01 01:00 abc
After:
1970-01-01 01:00 abc� def
|
|
Use putc instead of fputc, it can be optimized to macros.
From the OpenBSD man page:
" putc() acts essentially identically to fputc(), but is a macro that
expands in-line. It may evaluate stream more than once, so arguments
given to putc() should not be expressions with potential side effects."
sfeed_atom, sfeed_frames and sfeed_html are using this function.
Mini-benchmarked sfeed_html and it went from 1.45s to 1.0s with feed files in
total 250k lines (+- 350MB). Tested with clang and gcc on OpenBSD on an older
laptop.
|
|
|
|
iscntrl is c < ' ' || c == 127
I want to encode a space and everything above 127 also.
So this condition can be simplified to this.
|
|
- remove a check that has no use/can never happen.
- remove the return value as it's unused and the input size is known.
- fix an old comment that doesn't reflect what the function does anymore.
|
|
Optimize for the common-case: assuming ASCII.
The input is assumed to be valid UTF-8 input (output of sfeed).
This saves 2 function calls for determining the width of a single ASCII
character, which of course is 1.
Ranges:
< 32 are control-characters and are skipped.
< 127 is typical ASCII and is 1 column wide.
>= 127 is the normal path (control-character and multi-byte UTF-8).
Tested on OpenBSD and Linux with various compilers (clang, gcc, pcc and tcc).
On OpenBSD and Linux glibc much improvement. On Linux musl (almost) no change.
In a common-case upto 40% performance improvement.
In the worst-case negligible performance degration (<1%).
|
|
|
|
|
|
|
|
|
|
|
|
POSIX says about snprintf:
"If an output error was encountered, these functions shall return a
negative value".
So check for < 0 instead of -1. Afaik all implementations return -1 though.
|
|
|
|
|
|
- cast all ctype(3) function argument to (unsigned char) to avoid UB
POSIX says:
"The c argument is an int, the value of which the application shall ensure is a
character representable as an unsigned char or equal to the value of the macro
EOF. If the argument has any other value, the behavior is undefined."
Many libc cast implicitly the value, but NetBSD does not, which is probably the
correct thing to interpret it.
- no need to cast for putchar + rename some fputc(..., stdout) to putchar
POSIX says:
"The fputc() function shall write the byte specified by c (converted to an
unsigned char) to the output stream pointed to by stream [...]"
Major thanks to Leonardo Taccari <iamleot@gmail.com> for reporting and testing
it on NetBSD!
|
|
for example the string "\xef\xbc\xb5".
|
|
|
|
|
|
|
|
- use a UTF-8 ellipses (1 column width) for "...".
- do proper truncation at the specified length.
|
|
this entity is XHTML, it is not supported by some (older) browsers.
|
|
|
|
|
|
|
|
... as a bonus it also saves an allocation.
|
|
|
|
use long long: atleast 32-bit, but now time_t (real) to 32-bit or
64-bit is supported. Long long is C99 though, but that is fine.
check errno, it can have ERANGE.
|
|
|
|
|
|
- pledge tools and add define to enable it on platforms that support it, currently
only OpenBSD 5.9+
- separate getline and parseline functionality.
- use murmur3 hash instead of jenkins1: faster and less collisions.
- make some error messages a bit more clear, for example with path truncation.
- some small cleanups, move printutf8pad to util.
|
|
use the port specified in the link for urls starting with "//" (use protocol).
|
|
|
|
|
|
|
|
7f11ef506465896705f15c39bd0416d96ca651a8
|
