|100

Description:

  • Allows you to store, search, analyze, monitor, and alert on log data and events from Google Cloud and AWS.
  • Logging includes storage for logs, a user interface called Logs Explorer, and an API to manage logs programmatically.
  • can also be used in realtime using streaming like dataflow
  • Types of logs:
    • Platform log: Gcloud services
    • Component log: like platform log but generated by cloud-provided software components like GCE instances log
    • Security log: who did what where and when
    • User-written log: written by custom applications and services by API or client sdk
      • GKE, Run, Functions: log written to stdout and stderr are saved automatically
      • GCE: Install Ops Agent on VM
    • Multi/hybrid cloud log

1. Log explorer:

  • Query builder:
  • Make uses of the LHS log fields

2. Log analytics:

  • Powered by BigQuery
  • Enable analytics for the GCP Log Bucket
    • Different from export log to bq
    • Log data in BigQuery is managed by Cloud Logging.
    • BigQuery ingestion and storage costs are included in your Logging costs.
    • Data residency and lifecycle are managed by Cloud Logging.
    • Cant use join or other operations with other datasets in bq
  • Cant be downgrade to non-analytics bucket

3. Log dashboard:

4. Log-based metrics:

  • Derive data from content of log entries
  • 2 types:
    • System-defined log-based metrics:
      • Calculated only from logs that have been ingested by Cloud Logging
      • Can be: counter, distribution
      • At project level
    • User-defined log-based metrics:
      • Created by admin to track things
      • Can be: counter, distribution and boolean
      • can be project or bucket level
  • Labels are used for grouping and filtering with filed name
    • Allows for one time-series but multiple lines for each label
    • Can use extraction regular expression to extracts label value from field value
      • leave empty for the entire content of the field

5. Log router:

  • Can route from one project to a bucket of another project
  • GCP Log Sink
  • Log routing:
    • All types of logs is contralized in Cloud Logging API
    • Then send to 3 GCP Log Sink . Each contains the inclusion and exclusion filters
      • _reqrired log sink
      • _Default log sink
      • User-defined log sink
    • Then send to log buckets

6. Log storage:

  • Each project, logging automatically create
    • required log bucket and required log sink
      • Holds admin activity, system event and access transparency logs
      • Retention of 400 days, non-changeable
      • Cant be deleted or modified
    • default log bucket and default log sink
      • Holds all other logs excepts in required logs
      • Retention of 30 days by default, up to 365 days
      • Cant be deleted but can be disabled

7. Integration: