Stream Processing
Real-time data flows through the pipeline, enabling processors to handle incoming data promptly.
Building Streaming and Cross-Environment Data Processing Pipelines with RDF-Connect
Explore existing RDF-Connect processors, runners, orchestrators, and pipelines, contributed by developers across diverse use cases. Thanks to GitHub’s topic-based search, you can easily find and reuse what’s already available. Look for repositories tagged with rdf-connect and specific labels like rdfc-processor, rdfc-runner, rdfc-orchestrator, or rdfc-pipeline. If you’re contributing, tagging your work helps others discover and build upon it. Read more about discovering existing work.
An LDES client that can read a stream of members from an LDES.
rdf-connect/ldes-clientA processor that validates incoming RDF data against a SHACL shapes graph.
rdf-connect/shacl-processor-tsCollection of Typescript utilities around RML tools that includes a wrapper over the RML Mapper to be reused within the RDF-Connect ecosystem
rdf-connect/rml-utils-processor-tsTemplate processor to kickstart the development of your next TypeScript RDF-Connect processor.
rdf-connect/template-processor-tsWrites a LDES to local files
rdf-connect/sds-processors-tsTransform a stream of SDS wrapped members into a named graph-based member.
rdf-connect/sds-processors-tsHandle and pipe multiple input data streams into a single output stream.
rdf-connect/sds-processors-tsPipeline for the MuMo project, building a LDES from data generated by custom sensors
ajuvercr/mumo-pipelineRDF-Connect processor to harvest an LDES with some additional options to the ldes-client
smessie/ldes-harvester-processor-tsA processor that ingests RDF data into a SPARQL endpoint
rdf-connect/sparql-ingest-processor-tsTypescript util functions related to XML including a wrapper over the Saxon-HE library to be used within the RDF-Connect ecosystem
rdf-connect/xml-utils-processors-tsA simple Python processor to log an incoming stream to the RDF-Connect logging system.
rdf-connect/log-processor-pyA simple Python processor to send an array of messages to a channel.
rdf-connect/log-processor-pyTemplate processor to kickstart the development of your next Python RDF-Connect processor.
rdf-connect/template-processor-pyRDF-Connect processors for interacting with OpenStreetMap data
rdf-connect/osm-utils-processors-tsReads all files according to the Glob pattern
rdf-connect/file-utils-processors-tsReads all files inside the folder
rdf-connect/file-utils-processors-tsSimple string manipulation processor. Transform a stream by applying the string substitution on the messages. The matching string can be a regex.
rdf-connect/file-utils-processors-tsSubstitutes all declared environment variables to their respective values.
rdf-connect/file-utils-processors-tsReads a requested file on demand.
rdf-connect/file-utils-processors-tsProcessor to unzip a compressed file and stream out its content.
rdf-connect/file-utils-processors-tsProcessor to gunzip a compressed file and stream out its content.
rdf-connect/file-utils-processors-tsA processor for the RDF Connect framework that writes an incoming stream to disk and later reads it into the output stream.
rdf-connect/replication-processor-tsA processor that converts a dump of RDF data to a feed of RDF data based on a member SHACL shape.
rdf-connect/dumps-to-feed-processor-tsA simple JS/TS processor to log an incoming stream to the RDF-Connect logging system.
rdf-connect/log-processor-tsA simple JS/TS processor to send messages to a writer channel.
rdf-connect/log-processor-tsA processor that converts a RDF data to a skolemized form (removing blank nodes).
rdf-connect/skolemization-processor-tsRDF-Connect pipeline to ingest Vienna's weather forecast in a knowledge graph. This repository contains the incremental solutions for the hands-on coding session of the tutorial at SEMANTiCS 2025.
rdf-connect/vienna-weather-forecast-kg-pipelineRDF-Connect pipeline for materializing an LDES into a SPARQL graph store
rdf-connect/ldes2sparqlConnector Architecture-based pipeline to publish an LDES from the Blue Bike API
rdf-connect/CA-Blue-Bike-LDESRDF-Connect pipelines for producing and consuming a LDES from RINF data
rdf-connect/RDF-Connect-RINF-LDESPipeline for the MuMo project, building a LDES from data generated by custom sensors
ajuvercr/mumo-pipelineCollection of example pipelines for RDF-Connect grouped per programming language.
rdf-connect/example-pipelinesRDF-Connect pipeline to republish a content-wise mirrored LDES for the Marine Regions dataset
rdf-connect/marine-regions-ldes-mirror