Azure Event Hubs Configuration with Azure Schema Registry and Azure Blob Storage Example

This production‑ready configuration example is tailored for Microsoft Azure services. It uses the EventHubSender to integrate with Azure Event Hubs for event streaming and includes Azure Schema Registry for schema management. Large blob and slob data are stored in Azure Blob Storage.

This configuration example includes both file and console logging, suitable for production monitoring.

{
   "Configurations" : {
      "AzureSender" : {
         "Authentication" : {
           "Type" : "ConnectionString",
           "ConnectionString" :      
              "Endpoint=sb://<your‑eventhub‑namespace>.servicebus.windows.net/;
               SharedAccessKeyName=your-shared-access-policy;
               SharedAccessKey=your-key",
          }
       },
      "AzureSchemaRegistry" : {
         "Endpoint" : "https://<your-schema-registry>.servicebus.windows.net",
         "EventhubNamespace" : "<your-eventhub-namespace>", 
         "SchemaGroup" : "<your-eventhub-json-schema-group>",
         "SchemaFormat" : "Json",
         "Authentication" : {
           "Type" : "AzureClientSecret",
           "ClientId" : "<your-client-id>",
           "ClientSecret" : "<your-client-secret>",
           "TenantId" : "<your-tenant-id>"
         }
      },
      "FileReader" : {
         "CaptureDirectory" : "c:\\events\\capture-files",
         "Delay" : 1000,
         "RunContinuously" : true,
         "UndeliverableDirectory" : "c:\\events\\undeliverable"
      },
      "FileTopicManager" : {
         "TopicFileName" : "c:\\events\\config\\topic.config.json"
      },
      "Loggers" : [
         {
            "Delay" : 1000,
            "LogFileName" : "c:\\events\\logs\\event-producer-one.log",
            "MaxLevel" : "Info",
            "RotateKeep" : 10,
            "RotateSize" : 100,
            "Sink" : "File"
         },
         {
            "MaxLevel" : "Warn",
            "Sink" : "Console"
         }
      ],
      "AzureBlobStorageProvider" : {
         "Endpoint" : "https://<your-storage-account-name>.blob.core.windows.net,
         "BlobPrefix" : "events",
         "ContainerName" : "<event-lob-data>",
         "StagingDirectory" : "<staging-root-directory>",
         "Authentication" : {
            "Type" : "AzureLogin"
          }
       },
      "SizeThresholds" : {
         "BlobThresholdBytes" : "5MB",
         "SlobThresholdBytes" : "1MB"
      },
      "Catalog" : {
         "CatalogDirectory" : "C:\\Jade\\System\\Journals"
      },
      "SchemaCache" : {
        "CacheRootDirectory" : "C:\\Events\\SchemaCache"
      }
   },
   "Modules" : {
      "Reader" : "File",
      "Sender" : "Azure",
      "Serializer" : "Json",
      "StorageProvider" : "AzureBlobStorage",
      "TopicManager" : "File",
      "SchemaGenerator" : "Json",
      "SchemaRegistry" : "Azure"
   }
}

Key aspects of this Azure Event Hubs with Azure Schema Registry and Azure Blob Storage configuration example are listed in the following table.

Element	Description
AzureSender	Configured for Azure Event Hubs
AzureSchemaRegistry	Includes Azure Schema Registry configuration
Loggers	Contains two configurations: one for detailed file logging and another for console warnings
AzureBlobStorageProvider	Used for storing large blobs and slobs in Azure Blob Storage
SizeThresholds	Set to determine when to use external storage
Authentication	Contains two authentication types for each of the Azure services (for demonstration purposes only)