go-html-selector, v1.0

The library that implements collecting specified HTML tags and their attributes from an HTML document.

Major version.

Features

• collecting from an HTML document:
  • HTML tags;
  • HTML attributes;
• options:
  • filtering a result:
    • by specified HTML tags;
    • by specified HTML attributes;
• representing a result:
  • grouping HTML attributes by their tags;
• optimizations by the number:
  • of memory allocations;
  • of string copies.

Installation

Prepare the directory:

$ mkdir --parents "$(go env GOPATH)/src/github.com/thewizardplusplus/"
$ cd "$(go env GOPATH)/src/github.com/thewizardplusplus/"

Clone this repository:

$ git clone https://github.com/thewizardplusplus/go-html-selector.git
$ cd go-html-selector

Install dependencies with the dep tool:

$ dep ensure -vendor-only

Examples

htmlselector.SelectTags():

package main

import (
    "fmt"
    "log"
    "strings"

    htmlselector "github.com/thewizardplusplus/go-html-selector"
)

func main() {
    reader := strings.NewReader(`
        <ul>
            <li>
                <a href="http://example.com/1">1</a>
                <video
                    src="http://example.com/1.1"
                    poster="http://example.com/1.2">
                </video>
            </li>
            <li>
                <a href="http://example.com/2">2</a>
                <video
                    src="http://example.com/2.1"
                    poster="http://example.com/2.2">
                </video>
            </li>
        </ul>
    `)

    filters := []htmlselector.Filter{
        {
            Tag:        []byte("a"),
            Attributes: [][]byte{[]byte("href")},
        },
        {
            Tag:        []byte("video"),
            Attributes: [][]byte{[]byte("src"), []byte("poster")},
        },
    }

    tags, err := htmlselector.SelectTags(reader, filters)
    if err != nil {
        log.Fatal(err)
    }

    for _, tag := range tags {
        fmt.Printf("<%s>:\n", tag.Name)
        for _, attribute := range tag.Attributes {
            fmt.Printf("  %s=%q\n", attribute.Name, attribute.Value)
        }
    }

    // Output:
    // <a>:
    //   href="http://example.com/1"
    // <video>:
    //   src="http://example.com/1.1"
    //   poster="http://example.com/1.2"
    // <a>:
    //   href="http://example.com/2"
    // <video>:
    //   src="http://example.com/2.1"
    //   poster="http://example.com/2.2"
}

Benchmarks

htmlselector.SelectTags() with a simple markup:

BenchmarkSelectTags/simple_markup/10_tags/430B-8              200000         10419 ns/op         6736 B/op        49 allocs/op
BenchmarkSelectTags/simple_markup/100_tags/4.4K-8              20000         62599 ns/op        25408 B/op       412 allocs/op
BenchmarkSelectTags/simple_markup/1000_tags/45.7K-8             2000        583876 ns/op       190624 B/op      4015 allocs/op
BenchmarkSelectTags/simple_markup/10000_tags/476.3K-8            200       7063473 ns/op      3448441 B/op     40025 allocs/op
BenchmarkSelectTags/simple_markup/100000_tags/4.8M-8              20      77255459 ns/op     35420161 B/op    400035 allocs/op
BenchmarkSelectTags/simple_markup/1000000_tags/50.3M-8             2     758725576 ns/op    339752720 B/op   4000045 allocs/op

htmlselector.SelectTags() with a complex markup:

BenchmarkSelectTags/complex_markup/10_tags/1020B-8            100000         19263 ns/op        11216 B/op       151 allocs/op
BenchmarkSelectTags/complex_markup/100_tags/10.4K-8            10000        181784 ns/op        67280 B/op      1414 allocs/op
BenchmarkSelectTags/complex_markup/1000_tags/108.8K-8           1000       1813507 ns/op       740560 B/op     14019 allocs/op
BenchmarkSelectTags/complex_markup/10000_tags/1.1M-8             100      20045364 ns/op      9136367 B/op    140029 allocs/op
BenchmarkSelectTags/complex_markup/100000_tags/11.6M-8             5     221623379 ns/op     90800358 B/op   1400039 allocs/op
BenchmarkSelectTags/complex_markup/1000000_tags/120.6M-8           1    2162959416 ns/op    881045280 B/op  14000049 allocs/op

Repository

Link: https://github.com/thewizardplusplus/go-html-selector/tree/v1.0.

Content: code.

License: MIT.