TIL&TEL 23.01.16~23.01.22

TIL( Today I Learned)

23-01-16 월

• WORKPLACE

  • PHASE 1 9-12 | 6

    • isort

      • cli 실행과

      • file watch 실행이 다름

        • ## pyproject.toml 추가
          [tool.isort]
          profile = "black" 

  • PHASE 2 13-16 | 6

  • PHASE 3 16-18 | 4

23-01-17 화

• WORKPLACE

  • PHASE 1 9-12 | 6

    • 구축 필요한 테이블 준비 하기

  • PHASE 2 13-16 | 6

    • isec DB 서비스 로직내 테이블 <-> astron raw log 테이블 매칭

  • PHASE 3 16-18 | 4

23-01-18 수

• WORKPLACE

  • PHASE 1 9-12 | 6

    • jsonb load 이상해짐 디버깅

  • PHASE 2 13-16 | 6

    • API 보완

      • 데이터 없을때

  • PHASE 3 16-18 | 4

23-01-19 목

• WORKPLACE

  • PHASE 1 9-12 | 6

    • async pg jsonb 넣고 빼기 어떻게 되는지 확인해보기

  • PHASE 2 13-16 | 6

    • 테이블 검증

      • array_column::text::timestamp[]
      • array_column::text::text[]

  • PHASE 3 16-18 | 4

23-01-20 금

• WORKPLACE

  • PHASE 1 9-12 | 6

    • 중복 데이터 지우기..? 가져가서 데이터 중복 값 지우도록 할까?

      • 수집 테이블 읽을때, 중복 걸러서 읽도록

  • PHASE 2 13-16 | 6

    • 이상치 검출 주기적으로 동작 시키기

  • PHASE 3 16-18 | 4

TEL (Trial and Error Log)

23-01-16

• ERROR: Runtime exited with error: signal: killedRuntime.ExitErrorRequestId: 810a80d4-b9bd-40e8-b737-c8b2155b6625 Error: Runtime exited with error: signal: killed Runtime.ExitError

  • SITUATION:

    • aws lambda

  • REASON:

    • Max Memory Used: 320 MB

  • SOLUTION:

    • increase memory

23-01-18

• ERROR: asyncpg.exceptions.TooManyConnectionsError: sorry, too many clients already

  • SITUATION:

    • docker run

  • REASON:

    • this error comes from PostgreSQL, who normally limits 100 maximum concurrent connections by default

  • SOLUTION:

    • increase connection max from 100 to 1000
    • decrease pool connenction size 10 to 5