[XML] Diviser (ou éclater) un fichier XML

J'avais affaire à un fichier XML assez volumineux (~800Mo).

Pour le traiter j'avais besoin d'un outil pour éclater le fichier (split en anglais) en plusieurs petits fichiers...

J'ai donc installer l'outils xml_split à travers l'outils perl XML Twig (article lié : http://astunix.blogspot.fr/2016/03/centos-rhel-trouver-le-package-lie-une.html)

$ sudo yum install perl-XML-Twig-3.44-2.el7.noarch

Ensuite pour l'utiliser, c'est assez simple, il suffit de tapper :

xml_split fichier.xml

ou utiliser l'option de niveau si le résultat n'est pas intéressant :

xml_split -l 2 fichier.xml

Plus d'information :

xml_split(1) - Linux man page
Name

xml_split - cut a big XML file into smaller chunks

Description

"xml_split" takes a (presumably big) XML file and split it in several smaller files. The memory used is the memory needed for the biggest chunk (ie memory is reused for each new chunk).

It can split at a given level in the tree (the default, splits children of the root), or on a condition (using the subset of XPath understood by XML::Twig, so "section" or "/doc/section").

Each generated file is replaced by a processing instruction that will allow "xml_merge" to rebuild the original document. The processing instruction format is " ?>"

File names are -.xml, with -00.xml holding the main document.
Options

-l

    level to cut at: 1 generates a file for each child of the root, 2 for each grand child

    defaults to 1
-c
    generate a file for each element that passes the condition

    xml_split -c

will put each "section" element in its own file (nested sections are handled too)

    Note that at the moment this option is a lot slower than using "-l"
-s
    generates files of (approximately) . The content of each file is enclosed in a new element ("xml_split::root"), so it's well-formed XML . The size can be given in bytes, Kb, Mb or Gb.
-g
    groups elements in a single file. The content of each file is enclosed in a new element ("xml_split::root"), so it's well-formed XML .
-b
    base name for the output, files will be named -<.ext>

    is a sequence number, see below "--nb_digits" is an extension, see below "--extension"

    defaults to the original file name (if available) or "out" (if input comes from the standard input)
-n
    number of digits in the sequence number for each file

    if more digits than are needed, then they are used: if "--nb_digits 2" is used and 112 files are generated they will be named "-01.xml" to "-112.xml"

    defaults to 2
-e
    extension to use for generated files

    defaults to the original file extension or ".xml"
-i

use XInclude elements instead of Processing Instructions to mark where sub files need to be included

-v

verbose output
Note that this option can slow down processing considerably (by an order of magnitude) when generating lots of small documents
-V

outputs version and exit

-h

short help

-m

man (requires pod2text to be in the path)

Examples

xml_split foo.xml             # split at level 1
xml_split -l 2 foo.xml        # split at level 2
xml_split -c section foo.xml  # a file is generated for each section element
                              # nested sections are split properly

See Also

XML::Twig, xml_merge
Todo

optimize the code

    any idea welcome! I have already implemented most of what I thought would improve performances.
provide other methods that PIs to keep merge information
    XInclude is a good candidate (alpha support added in 0.04).

    using entities, which would seem the natural way to do it, doesn't work, as they make it impossible to have both the main document and the sub docs to be well-formed if the sub docs include sub-sub docs (you can't have entity declarations in an entity)

Author

Michel Rodriguez
License

This tool is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

Source : http://search.cpan.org/dist/XML-Twig/tools/xml_split/xml_split

[JAVASCRIPT] XML <->JSON (xml2json)

L'outils XML2JSON permet de convertir des fichier XML en JSON ce qui peut être pratique dans

certains cas de figures.
English Title: "How to convert  XML to JSON and JSON to XML in CLI"

les sources de xml2json https://www.npmjs.com/package/xml2json

Installation

$ npm install xml2json
npm http GET https://registry.npmjs.org/xml2json
npm http 200 https://registry.npmjs.org/xml2json
npm http GET https://registry.npmjs.org/xml2json/-/xml2json-0.9.0.tgz
npm http 200 https://registry.npmjs.org/xml2json/-/xml2json-0.9.0.tgz
[...]

Un répertoire nommé "node_modules" se créé là où vous avez exécuter la commande

Utiliser xml2json pour transformer du xml en json

Tout d'abord localisons xml2json: Il se trouve dans node_modules/bin et taper la commande sous la forme suivante:

$ cat monfichier.xml  | /home/stan/node_modules/xml2json/bin/xml2json > monfichier.json

Le fichier JSON est alors généré

On peut alors utiliser jq (https://stedolan.github.io/jq/tutorial/) pour bien formater le JSON.

Utiliser xml2json pour transformer du json en xml

Là ça se complique ! il y a bien un json2xml, mais pas de binaire pour l’exécuter, juste des libs
Je les ai quand même installées:

Installation

# npm install xml2json

Ensuite aller dans ~/node_modules/xml2json/bin/ (oui je dis bien dans xml2json et pas json2xml)

et créer le fichier json2xml

$ vim json2xml

Contenu du fichier:

#!/usr/bin/env node

var json2xml = require('../');
var pkg = require('../package.json');

var json = '';

var args = process.argv.slice(2)
var arg = args[0]

if (arg == '--version') {
        console.log(pkg.version)
        process.exit(0)
}

process.stdin.on('data', function (data) {
        json += data;
});

process.stdin.resume();

process.stdin.on('end', function () {
        xml = json2xml.toXml(json)
        process.stdout.write(xml + '\n')
});

Enregistrer et quitter, puis rendez le exécutable:

$ chmod +x json2xml
puis l'executer:
$ cat $JSONFILE | /opt/tools/node_modules/xml2json/bin/json2xml > $XMLFILE

 Et voilà