Arcsight Platform and database

Hello,
do I understand correctly I need setup communal storage on the cloud ? Documentation saying "Communal storage is based on an object store, such as Amazon's S3 bucket in the cloud or a storage device for an on-premises deployment. The database relies on the object store to maintain the durable copy of the data." However, Arcsight DB installation has only these selection.

# ========================================
# STEP 2: Specify communal storage details
Supported communal storage types -
1) S3
2) Azure Blob Storage
3) Google Cloud Storage
Choose a communal storage type from the above (1-3):

Regards
Jan

  • 0

    Hello Jan, 

    can you please clarify where you are trying to deploy the platform in the cloud or on-premises and with what capabilities?

    Best Regards, 

    Daniel

  • 0 in reply to 

    Hi,
    it is on-premise install and I am want to try SOAR, and later implement TH (using soar,th,fusion yaml). In reality we have 250 EPS in ESM from Logger, I presume we do not need big and complex database.

  • 0 in reply to 

    Hello Jan, 

    S3 bucket is needed only if you are implementing Recon.

    For TH, SOAR, and Fusion configuring an S3 bucket is not needed.

    I have no idea what version Platform version you are but for example in 24.2 to add other capabilities to an existing cluster off the cloud  follow these steps: https://www.microfocus.com/documentation/arcsight/arcsight-platform-24.2/arcsight-pag-oc-24.2/#cluster_add/cluster_deploy.htm?TocPath=Adding%2520Additional%2520Capabilities%2520to%2520an%2520Existing%2520Cluster%257C_____2

    You may look over "/opt/arcsight/arcsight-platform-installer-XXXX/config" yaml examples files to adjust as you need.

    Also, information about the yaml files can be found here as well: https://www.microfocus.com/documentation/arcsight/arcsight-platform-24.2/arcsight-pag-oc-24.2/#deployment_offcloud_auto/example_config_files.htm?TocPath=Creating%2520an%2520Off-cloud%2520Deployment%257CPreparing%2520Your%2520Environment%257C_____7

    Hopefully, this will help you install SOAR.

    Best Regards, 

    Daniel

  • 0 in reply to 

    Thank you,
    I though the problem with fusion pods after install is due to missing database.

          

    Status of pods
    
    NAMESPACE                  NAME                                                     READY   STATUS      RESTARTS         AGE
    arcsight-installer-926qc   arcmc-update-io-license-nch7x                            0/1     Completed   0                65m
    arcsight-installer-926qc   autopass-lm-69f6cc86b5-6f652                             2/2     Running     4 (10m ago)      65m
    arcsight-installer-926qc   fusion-arcmc-web-app-cbbb785f8-hn2kq                     0/4     Pending     0                65m
    arcsight-installer-926qc   fusion-arcsight-configuration-service-77fff57686-ttcfs   2/2     Running     6 (10m ago)      65m
    arcsight-installer-926qc   fusion-common-doc-web-app-6c96fd77cf-wk8wr               2/2     Running     4 (10m ago)      65m
    arcsight-installer-926qc   fusion-dashboard-web-app-b586c899-b8kft                  2/2     Running     7 (10m ago)      65m
    arcsight-installer-926qc   fusion-db-adm-schema-mgmt-777f878d8b-5mpgj               0/1     Init:1/2    0                65m
    arcsight-installer-926qc   fusion-db-monitoring-web-app-6f466bfb54-4bnc7            0/2     Init:3/4    0                65m
    arcsight-installer-926qc   fusion-db-search-engine-68fdd67449-xs7l9                 0/2     Init:2/3    0                65m
    arcsight-installer-926qc   fusion-metadata-rethinkdb-659769b4bb-cjx5l               1/1     Running     2 (10m ago)      65m
    arcsight-installer-926qc   fusion-metadata-web-app-588997bdd8-jglbp                 2/2     Running     5 (10m ago)      65m
    arcsight-installer-926qc   fusion-reporting-web-app-5b5796489-sx498                 0/2     Pending     0                65m
    arcsight-installer-926qc   fusion-search-and-storage-web-app-59bf8888f4-xg2th       0/2     Init:1/5    0                65m
    arcsight-installer-926qc   fusion-search-web-app-bfbd445bc-hn4ss                    0/2     Init:1/4    0                65m
    arcsight-installer-926qc   fusion-single-sign-on-67c5df4989-ntcn6                   2/2     Running     4 (10m ago)      65m
    arcsight-installer-926qc   fusion-ui-services-5c7c679755-cqt82                      2/2     Running     4 (10m ago)      65m
    arcsight-installer-926qc   fusion-user-management-658994746c-gx6mv                  1/2     Running     7 (3m21s ago)    65m
    arcsight-installer-926qc   layered-analytics-widgets-5f6f789cc6-xgwc4               1/1     Running     2 (10m ago)      65m
    arcsight-installer-926qc   nginx-ingress-controller-v9pbz                           1/1     Running     2 (10m ago)      65m
    arcsight-installer-926qc   soar-frontend-6b684cfc68-q5vtz                           2/2     Running     4 (10m ago)      65m
    arcsight-installer-926qc   soar-gateway-57c59795f7-zk877                            2/2     Running     4 (10m ago)      65m
    arcsight-installer-926qc   soar-message-broker-7d7d6dc4df-gll8k                     2/2     Running     4 (10m ago)      65m
    arcsight-installer-926qc   soar-web-app-7b6bff8849-wfczt                            1/2     Running     9 (19s ago)      65m
    arcsight-installer-926qc   suite-reconf-pod-arcsight-installer-86c8dbb798-gg7s4     2/2     Running     4 (10m ago)      65m
    
    logs from one Init pods
    
      Type     Reason                  Age                From               Message
      ----     ------                  ----               ----               -------
      Normal   Scheduled               66m                default-scheduler  Successfully assigned arcsight-installer-926qc/fusion-search-web-app-bfbd445bc-hn4ss to arcsightplatform.server.com
      Warning  FailedScheduling        67m                default-scheduler  0/1 nodes are available: 1 node(s) didn't match Pod's node affinity/selector. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
      Normal   Pulled                  66m                kubelet            Container image "localhost:5000/arcsight/kubernetes-vault-init:0.20.0-0013" already present on machine
      Normal   Created                 66m                kubelet            Created container install
      Normal   Started                 66m                kubelet            Started container install
      Normal   Pulled                  66m                kubelet            Container image "localhost:5000/arcsight/arcsight-alpine:3.18.6" already present on machine
      Normal   Created                 66m                kubelet            Created container dependence-hercules-search-engine
      Normal   Started                 66m                kubelet            Started container dependence-hercules-search-engine
      Warning  FailedCreatePodSandBox  45m                kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "001a2a1bdf44ef6d45ed243fd6caff9d84f2f0bcfeddd302a60fc4d014f206e2": plugin type="flannel" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory
      Normal   Started                 45m                kubelet            Started container install
      Normal   Pulled                  45m                kubelet            Container image "localhost:5000/arcsight/kubernetes-vault-init:0.20.0-0013" already present on machine
      Normal   Created                 45m                kubelet            Created container install
      Normal   SandboxChanged          45m (x2 over 45m)  kubelet            Pod sandbox changed, it will be killed and re-created.
      Normal   Pulled                  44m                kubelet            Container image "localhost:5000/arcsight/arcsight-alpine:3.18.6" already present on machine
      Normal   Created                 44m                kubelet            Created container dependence-hercules-search-engine
      Normal   Started                 44m                kubelet            Started container dependence-hercules-search-engine
      Warning  FailedMount             16m                kubelet            MountVolume.SetUp failed for volume "kube-api-access-nskm6" : failed to sync configmap cache: timed out waiting for the condition
      Warning  FailedCreatePodSandBox  12m                kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "c0ad286c53045fd5d785ee83747aa39caa276d82e360960f9ae21dace4e4bfe6": plugin type="flannel" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory
      Normal   SandboxChanged          12m (x2 over 12m)  kubelet            Pod sandbox changed, it will be killed and re-created.
      Normal   Pulled                  12m                kubelet            Container image "localhost:5000/arcsight/kubernetes-vault-init:0.20.0-0013" already present on machine
      Normal   Created                 12m                kubelet            Created container install
      Normal   Started                 12m                kubelet            Started container install
      Normal   Pulled                  12m                kubelet            Container image "localhost:5000/arcsight/arcsight-alpine:3.18.6" already present on machine
      Normal   Created                 12m                kubelet            Created container dependence-hercules-search-engine
      Normal   Started                 12m                kubelet            Started container dependence-hercules-search-engine
    
    YAML used to install
    
    # ArcSight SOAR single node installation configuration
    #
    # This example configures all components required by SOAR to be installed on a single node including Recon, Transformation Hub, and Database.
    #
    # Before using this example, you must do the following:
    # - Replace the hostname "yourdomain-node.yourenterprise.net" (same value in multiple locations) with the correct values for your environment.
    # - Set the values of arcmc-generator-id-start and arcmc-generator-id-end to form a valid Generator ID Range
    #     Generator ID Range: set a valid range within [1-16383].
    #     The difference between the values must be more than 100.
    #
    # Key aspects of this configuration are:
    # - Encryption: yes
    # - Transformation Hub TLS Client Auth: yes
    # - FIPS 140 mode: yes
    # - High Availability: no, since it is deployed on a single node
    #
    cluster:
      default-node-size: small
      allow-worker-on-master: true
      enable-fips: false
      master-nodes:
        - hostname: arcsightplatform.server.com
          username: root
          labels: [fusion]
    
    suite:
      products: [soar]
      config-params:
        # Transformation Hub, defaults are commented
        # th-init-fips: true
        # th-init-noOfTopicPartitions: 6         
        # th-init-kafkaRetentionBytes: When the database is colocated with the Kubernetes cluster, the ArcSight Platform Installer automatically calculates this value.
        #th-schema-registry-count: 1
        #transform-processor-replicas: 2
        #enrichment-processor1-replicas: 2
        #th-enrichment-processor-integrity-enabled: false
        #routing-processor1-replicas: 0
        #th-kafka-count: 1
        #th-zookeeper-count: 1
        #th-init-kafkaOffsetsTopicReplicationFactor: 1
        #th-init-topicReplicationFactor: 1
        #th-kafka-allow-plaintext: false
        #th-init-client-auth: true
        #Core
        #Generator ID parameters are mandatory parameters.
        #Ensure that you set arcmc-generator-id-enable to true.
        arcmc-generator-id-start: 12200
        arcmc-generator-id-end: 12500
        arcmc-generator-id-enable: true
     

  • Verified Answer

    +1 in reply to 

    Hello Jan,

    for a single node with Fusion and SOAR, for example, look at example-install-config-esm_cmd_center-single-node.yaml

    .......

    cluster:
          default-node-size: small
          allow-worker-on-master: true
          enable-fips: false
           master-nodes:
                - hostname: yourdomain-node.yourenterprise.net
                  username: root
                  labels: [fusion]

    suite:  
       products: [soar]
       config-params:
           # Fusion
           search-engine-replica: 0
           vertica-enable: false

    # Fusion
    # Generator ID parameters are mandatory parameters.
    # Ensure that you set arcmc-generator-id-enable to true.
            arcmc-generator-id-start: 1
            arcmc-generator-id-end: 16383
            arcmc-generator-id-enable: true

    .......

    that should be enough.

    You get it right but you forgot to add "vertica-enable: false" and tried to add also "search-engine-replica: 0" , no idea since I have not tested this scenario yet with 24.2 but try to see if it still accepting it in the yaml file.

    In this way, you have all those pods related to DB will not appear anymore.

    Best Regards, 

    Daniel

  • 0 in reply to 

    Thank you, I have tried this and it worked. Settings "search-engine-replica: 0" is no more 24.2, I have used search-engine-replica: 0.