Skip to content

fix(aws_s3 source): add retry delay in sqs::Ingestor #22999

New issue

Have a question about this project? Sign up for a free account to open an issue and contact its maintainers and the community.

By clicking “Sign up for ”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on ? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

medzin
Copy link

@medzin medzin commented May 6, 2025

Summary

This commit adds logic to wait 500ms between subsequent calls of run_once if it returns an error. This is required to prevent spamming the SQS API if it fails to receive messages, e.g. due to invalid IAM permissions or an SQS outage.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

How did you test this PR?

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the "no-changelog" label to this PR.

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • The CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • cargo fmt --all
      • cargo clippy --workspace --all-targets -- -D warnings
      • cargo nextest run --workspace (alternatively, you can run cargo test --all)
      • ./scripts/check_changelog_fragments.sh
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run cargo v licenses to regenerate the license inventory and commit the changes (if any). More details here.

References

@medzinmedzin requested a review from a team as a code owner May 6, 2025 21:26
@bits-bot
Copy link

bits-bot commented May 6, 2025

CLA assistant check
All committers have signed the CLA.

@github-actions-actions bot added the domain: sourcesAnything related to the Vector's sourceslabel May 6, 2025
This commit adds logic to wait 500ms between subsequent calls of
run_once if it returns an error. This is required to prevent spamming
the SQS API if it fails to receive messages, e.g. due to invalid IAM
permissions or an SQS outage.
Sign up for free to join this conversation on . Already have an account? Sign in to comment
Labels
domain: sourcesAnything related to the Vector's sources
Projects
None yet
Development

Successfully merging this pull request may close these issues.

No retry backoff for sqs in aws_s3 source
2 participants