TIL&TEL 22.09.19~22.09.25

TIL( Today I Learned)

22-09-19 월

• Event

  • 텔라 통화

    • \1. What other companies did you apply to? Why would you choose our company over them?

      • I have applied to Greenlaps.
      • These ary why I choose your company..
      • Your business model is attractive. It has huge market and your product has a lot of chane to scale up.
      • I want to contribute scale the business through my software skills and develop my career

    • \2. Why do you think you are fit for the position that you are applying for?

      • I hava been not only in the software area but also on the business, so I can understand business logic better and contribute the business
      • soft skills
      • hard

    • \3. What tasks are you expecting to fulfill if you start working for us?

      • I like robust and flexible software. So make your software more Reliabl than before by adding safety tools
      • And I gonna make your business request become software as you wish.

    • \1. How would your friends describe you?

      • My friends say I am smart and quick learner and deliberate for works.

    • \2. What can you offer to the company?

      • I can give you more reliable and flexible software.
      • It means it deliver nice values to your custome and also you.
      • by making customers problem and your requested feature are made

    • \3. What do you think the biggest problem of our company is, and how would you improve it?

      • I can tell you 3 problems you

      • Process getting older and older

        • Too much meetings with out specific Goals

• WORKPLACE

  • PHASE 1 9-12 / 6

    • parameter 불러와서 DB conn 부르기
    • 부를 수 있는 정보

  • PHASE 2 13-16 / 6

    • logging 방법 논의

  • PHASE 3 16-18 / 4

22-09-20 화

• WORKPLACE

  • PHASE 1 9-12 / 6

    • 결과물 부가 데이터 정리하기

      • 검사 필터명
      • 검사 데이터
      • 결과
      • discriminate
      • detect

    • 데이터 검사법 성능 이슈 대응

      • 상황:

        • 필터 데이터 is in Dict
        • 신규 데이터 In Dataframe or tuple( from DB)

      • 해결법

        1. dict.get(“rowdatacolumn”) O(n) * O(1)
        2. df.isin( 필터데이터 풀어놓는 칼럼 ) O(n) 예상
        3. df.isin( 필터데이터 dict.keys() ) O(n) 예상

    • 엔진 필요

      • db connector 장착 필요
      • schema 추가 해야함
  • PHASE 2 13-16 / 6

  • PHASE 3 16-18 / 4

    • logging 방법 논의

22-09-21 수

• WORKPLACE

  • PHASE 1 9-12 / 6

    • 수집 db 연결 확인

    • parameter 전달 확인
    • 감시모듈 리뷰

  • PHASE 2 13-16 / 6

    • cloud trail 배치 프로세스
    • 검사 방법 검증하기

  • PHASE 3 16-18 / 4

    • logging 방법 레퍼런스

22-09-22 목

• WORKPLACE

  • PHASE 1 9-12 / 6

    • logging 방법 레퍼런스

      • basic
      • debug 두개로 구분

      • root 로거 한개로 처리

        • dictConf

  • PHASE 2 13-16 / 6

    • cloud trail 배치 프로세스
    • logger, connector 정보 전달 1|

  • PHASE 3 16-18 / 4

22-09-23 금

• WORKPLACE

  • PHASE 1 9-12 / 6

    • batch 모드 돌리기

TEL (Trial and Error Log)

22-09-19

• ERROR: ImportError: cannot import name

  • SITUATION:

    • module import 하는데 안됨

  • REASON:

    • init.py 에서 동일한 이름을 사용하고 있어서 어떤건지 알수 가 없었다.

  • SOLUTION:

    • 중복되는 네임 제거하기

• ERROR: DB connection 생성 안됨

  • SITUATION:

    • db conn 테스트

  • REASON:

    • self 주소 접근시, 외부 ip를 호출하고 있었음

  • SOLUTION:

    • 서버 내부에서 db 호출할땐 127.0.0.1 로 부르기

• ERROR: SettingWithCopyWarning:

  • A value is trying to be set on a copy of a slice from a DataFrame.
  • Try using .loc[rowindexer,colindexer] = value instead

  • SITUATION:

    • pandas data frame editing filtered data

  • REASON:

    • df 슬라이싱 데이터 변경시, 원본에는 변경 되지 않음을 경고함

  • SOLUTION:

    • target.loc[target[col].isin(groupby[col])][f"pass_{col}"] = True<= target[target[col].isin(groupby[col])][f"pass_{col}"] = True

• ERROR: TypeError: Cannot perform ‘rand_’ with a dtyped [bool] array and scalar of type [bool]

  • SITUATION:

    • dataframe filtering with &

  • REASON:

    • multi condition 에서 괄호 빼먹었음

  • SOLUTION:

    • 괄호를 잘 매기자

22-09-20

• ERROR: str 조건문 통과함

  • SITUATION: str 조건문 통과함
  • REASON: str compare str 비교에 is 사용

  • SOLUTION:

    • != <== is not

• ERROR: AttributeError: module ‘sqlalchemy.dialects’ has no attribute ‘postgresql’

  • SITUATION:

    • module import
  • REASON:

  • SOLUTION:

    • from sqlalchemy.dialects import postgresql; <= import sqlalchemy.dialects

• ERROR: AttributeError: ‘InOrNotDetector’ object has no attribute ‘name’

  • SITUATION:

    • class 이름 가져오기

  • REASON:

    • instance attr 에 __name__ 없음

  • SOLUTION:

    • instance.__class__.__name__ 으로

• ERROR: raise ValueError(f”{col} ({my_type}) not a string”)

  • SITUATION:

    • pandas.to_sql()

  • REASON:

    • to 일반 connection 객체 말고 sqlalchemy Engine만 받는다.

  • SOLUTION:

    • Sa.engine <== conn.conn

• ERROR: ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

  • SITUATION:

    • dataframe empty check

  • REASON:

    • bool(df) 지원 하지 않음

  • SOLUTION:

    • `if df.empty: print(‘DataFrame is empty!’)`

22-09-22

• ERROR: root config is not inherit

  • SITUATION:

    • python logging dict config applying
    • 라인별로 선언하면 잘 되는데, dict config 적용하면 잘 안된다.

  • REASON:

    • disableexistingloggers option이 다른것 꺼버림

  • SOLUTION:

    • {...
        "disable_existing_loggers"= False}