Troubleshooting

This troubleshooting guide includes step-by-step instructions that should resolve most installation issues.

Using Troubleshooting repository

You can use the SigNoz troubleshoot to test if SigNoz otel collector is accessible from where you are running your applications.

Binary installation

You can run the one liner script below to download the troubleshoot binary:

curl -sL https://github.com/SigNoz/troubleshoot/raw/main/scripts/install.sh | bash

Here is the syntax:

./troubleshoot checkEndpoint --endpoint=<endpoint-to-check>

Binary

For example, if Otel Collector should be accessible in localhost:4317:

./troubleshoot checkEndpoint --endpoint=localhost:4317

Docker

You can also use troubleshoot Docker image:

docker run -it --rm signoz/troubleshoot checkEndpoint --endpoint=172.17.0.1:4317

*Notes: SigNoz Otel Collector should be accessible in 172.17.0.1:4317 from your application even if running in different docker network.

Kubernetes

Let's spin up a pod in Kubernetes with platform namespace to check if otel collector is running properly:

kubectl -n platform run troubleshoot --image=signoz/troubleshoot \
  --restart='OnFailure' -i --tty --rm --command -- ./troubleshoot checkEndpoint \
  --endpoint=my-release-signoz-otel-collector.platform.svc.cluster.local:4317

You can also spin up a pod in Kubernetes with same namespace as your application to check if otel collector is accessible:

kubectl -n app-namespace run troubleshoot --image=signoz/troubleshoot \
  --restart='OnFailure' -i --tty --rm --command -- ./troubleshoot checkEndpoint \
  --endpoint=my-release-signoz-otel-collector.platform.svc.cluster.local:4317

*Notes:

1. Replace app-namespace with your application namespace, my-release with SigNoz helm release name, and platform with SigNoz namespace.
2. In case on multiple k8s cluster, you might have to set otel collector service type as NodePort or LoadBalancer.

helm upgrade --install -n platform my-release signoz/signoz \
  --set otelCollector.serviceType="<NodePort or LoadBalancer>"

 

Troubleshooting Video

Whew! That was a lot of instruction to follow. If you instead prefer to watch a video, here you go

Docker Standalone

1. Before you install SigNoz on Docker Standalone, ensure that all prerequisites are met.
2. Run the docker ps command and ensure that the clickhouse-setup_clickhouse and query-service containers are running. If these containers are not running, increase the memory allocated to Docker.
3. If you don't see any data in the front-end, please wait two or three minutes.
4. Run the install.sh script again. The script will try to reinstall the containers that failed.
5. Follow the steps for uninstalling SigNoz section and then install SigNoz again by following the steps in the Install SigNoz on Docker Standalone section.
6. If you're still facing issues trying to install SigNoz, please reach out to us on Slack

SigNoz Otel Collector address Grid

You might have specific set up for your application and SigNoz cluster. It might not be very clear on which address to use to send data to SigNoz.

Here is the SigNoz Otel Collector address grid which could be helpful:

		Where SigNoz is installed?
		VM (Docker) - Same Machine	VM (Docker) - Different Machine	K8s (Same Cluster)	K8s (Different Cluster)
Where your application is running?	VM (native/binary)	localhost:4317	<otelcollector-IP>:4317	<k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317	<k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317
	VM (Docker)	172.17.0.1:4317, otel-collector:4317(shared network)	<otelcollector-IP>:4317	<k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317	<k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317
	Kubernetes	<otelcollector-IP>:4317	<otelcollector-IP>:4317	<release-name>-signoz-otel-collector.<namespace>.svc.cluster.local:4317	<k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317

Notes:

1. For the <otelcollector-IP>, use private IP address if the VM is in same private network.
2. Replace <namespace> with SigNoz namespace and <release-name> with SigNoz helm release name.
3. In the case of k8s where the application and SigNoz are running in different k8s cluster, you will have to expose otel collector service. Set the service type to either NodePort or LoadBalancer.

   helm upgrade --install -n platform my-release signoz/signoz \
   --set otelCollector.serviceType="<NodePort or LoadBalancer>"