go-html-selector, v1.5

The library that implements collecting specified HTML tags and their attributes from an HTML document.

Support of the universal tag (*).

Change Log

filtering a result:
- support of the universal tag (*);
optimizations:
- removing duplicate attribute filters, if there are the same filters for the universal tag.

Features

collecting from an HTML document:
- HTML tags;
- HTML attributes;
options:
- filters:
  - filtering a result:
    - by specified HTML tags:
      - with support of the universal tag (*);
    - by specified HTML attributes;
  - friendly representation of filters:
    - for parsing from JSON;
    - for definition as a code literal;
- skipping empty entities (separately optional):
  - tags without attributes;
  - attributes with an empty value;
representing a result:
- using the builder interface for building a result;
- built-in builders:
  - with grouping HTML attributes by their tags;
  - with collecting only values of HTML attributes;
optimizations:
- of searching for a right filter among others:
  - with removing duplicate attribute filters, if there are the same filters for the universal tag;
- of conversion from a byte slice to a string;
- by the number:
  - of memory allocations;
  - of string copies.

Examples

htmlselector.SelectTags() with the universal tag:

package main

import (
    "fmt"
    "log"
    "strings"

    htmlselector "github.com/thewizardplusplus/go-html-selector"
    "github.com/thewizardplusplus/go-html-selector/builders"
)

func main() {
    reader := strings.NewReader(`
        <ul>
            <li>
                <a href="http://example.com/1" title="link #1">1</a>
                <video
                    src="http://example.com/1.1"
                    poster="http://example.com/1.2"
                    title="video #1">
                </video>
            </li>
            <li>
                <a href="http://example.com/2" title="link #2">2</a>
                <video
                    src="http://example.com/2.1"
                    poster="http://example.com/2.2"
                    title="video #2">
                </video>
            </li>
        </ul>
    `)

    filters := htmlselector.OptimizeFilters(htmlselector.FilterGroup{
        htmlselector.UniversalTag: {"title", "href"},
        "video":                   {"src", "poster"},
    })

    var builder builders.StructuralBuilder
    err := htmlselector.SelectTags(
        reader,
        filters,
        &builder,
        htmlselector.SkipEmptyTags(),
    )
    if err != nil {
        log.Fatal(err)
    }

    for _, tag := range builder.Tags() {
        fmt.Printf("<%s>:\n", tag.Name)
        for _, attribute := range tag.Attributes {
            fmt.Printf("  %s=%q\n", attribute.Name, attribute.Value)
        }
    }

    // Output:
    // <a>:
    //   href="http://example.com/1"
    //   title="link #1"
    // <video>:
    //   src="http://example.com/1.1"
    //   poster="http://example.com/1.2"
    //   title="video #1"
    // <a>:
    //   href="http://example.com/2"
    //   title="link #2"
    // <video>:
    //   src="http://example.com/2.1"
    //   poster="http://example.com/2.2"
    //   title="video #2"
}

Benchmarks

With Structural Builder

htmlselector.SelectTags() with the universal tag:

BenchmarkSelectTags/structural_builder/universal_tag/10_tags/1020B-8               50000         25557 ns/op        18370 B/op       126 allocs/op
BenchmarkSelectTags/structural_builder/universal_tag/100_tags/10.4K-8              10000        311815 ns/op       168108 B/op      1206 allocs/op
BenchmarkSelectTags/structural_builder/universal_tag/1000_tags/108.8K-8             1000       2407620 ns/op      1437633 B/op     12006 allocs/op
BenchmarkSelectTags/structural_builder/universal_tag/10000_tags/1.1M-8               100      22065997 ns/op     11180076 B/op    120006 allocs/op
BenchmarkSelectTags/structural_builder/universal_tag/100000_tags/11.6M-8              10     445495615 ns/op    244945105 B/op   1200006 allocs/op
BenchmarkSelectTags/structural_builder/universal_tag/1000000_tags/120.6M-8             1    2307457875 ns/op    383996112 B/op  12000006 allocs/op

With Flatten Builder

htmlselector.SelectTags() with the universal tag:

BenchmarkSelectTags/flatten_builder/universal_tag/10_tags/1020B-8                 100000         17471 ns/op         9823 B/op        46 allocs/op
BenchmarkSelectTags/flatten_builder/universal_tag/100_tags/10.4K-8                 10000        140121 ns/op        57790 B/op       406 allocs/op
BenchmarkSelectTags/flatten_builder/universal_tag/1000_tags/108.8K-8                1000       1577272 ns/op       802345 B/op      4006 allocs/op
BenchmarkSelectTags/flatten_builder/universal_tag/10000_tags/1.1M-8                  100      15151013 ns/op      5776548 B/op     40006 allocs/op
BenchmarkSelectTags/flatten_builder/universal_tag/100000_tags/11.6M-8                 10     164593709 ns/op     70617641 B/op    400006 allocs/op
BenchmarkSelectTags/flatten_builder/universal_tag/1000000_tags/120.6M-8                1    1620865858 ns/op    869134992 B/op   4000007 allocs/op

Repository

Link: https://github.com/thewizardplusplus/go-html-selector/tree/v1.5.

Content: code.

License: MIT.